Deepa, For the process that get killed, you can check the kernel message with dmesg to see if it is memory related. They might be getting killed by the Linux OOM killer. The segfault could also be related to a failing memory allocation.
Thanks, From: [email protected] [mailto:[email protected]] On Behalf Of Deepa Sahni Sent: Friday, May 25, 2012 4:50 PM To: K. John Wu Cc: FastBit Users Subject: Re: [FastBit-users] ibis : select query causing segmentation fault and process killed The number of rows in one million. I used ardea utility to load the data from a .csv file. Examples of queries I ran are given below(I am including the output because it includes the query too ). Although from what I could see any query with these columns in the where clause is failing. ./benchmark_q2.sh: line 46: 1923 Killed /home/deepa/Downloads/fastbit-ibis1.3.0/examples/ibis -d /home/deepa/Downloads/fastbit-ibis1.3.0/tmp -q "select count(*) where K2=2 and K4=3" ./benchmark_q2.sh: line 48: 1963 Killed /home/deepa/Downloads/fastbit-ibis1.3.0/examples/ibis -d /home/deepa/Downloads/fastbit-ibis1.3.0/tmp -q "select count(*) where K2=2 and K10=3" ./benchmark_q2.sh: line 50: 2019 Segmentation fault /home/deepa/Downloads/fastbit-ibis1.3.0/examples/ibis -d /home/deepa/Downloads/fastbit-ibis1.3.0/tmp -q "select count(*) where K2=2 and K100=3" Also if you think it does need more memory, could you give some specifics. In the sense if I want to fix the issues how much more memory should I make available to it. Does it need more main memory or hard disk space? Thanks, Deepa ./benchmark_q2.sh: line 52: 2047 Segmentation fault /home/deepa/Downloads/fastbit-ibis1.3.0/examples/ibis -d /home/deepa/Downloads/fastbit-ibis1.3.0/tmp -q "select count(*) where K2=2 and K10K=3" On Fri, May 25, 2012 at 1:20 PM, K. John Wu <[email protected]<mailto:[email protected]>> wrote: Hi, Deepa, I would guess that you are using data generated by setqgen. Would you mind tell us how many rows are there? What are the exact queries you've used? I would assume that your join is a little too big to be held in 1GB of memory. If you give me the about information, I would be able to confirm this speculation. John On 5/25/12 1:07 PM, Deepa Sahni wrote: > > Hi John, > > I have attached the data partition details of all the columns. The > issue I am having is with the following columns. > > Column Name Issue > -------------------------------------------------- > k4 killed > k10 killed > k10k segmentation fault > k100 segmentation fault > > In addition to this I am running FastBit on ubuntu(running on top of > VMPlayer) and the ram available to it is 1GB and hard disk is 20GB. > If you need any other information I will provide that too. > > Thanks, > Deepa > > > > On Fri, May 25, 2012 at 9:36 AM, K. John Wu <[email protected]<mailto:[email protected]> > <mailto:[email protected]<mailto:[email protected]>>> wrote: > > Yes, you are probably encounter memory shortage. Since you did not > give any size information, there is no way for me to verify your > statement;-) > > John > > > On 5/25/12 1:09 AM, Deepa Sahni wrote: > > Hi, > > > > I need some help debugging an issue. FYI I am running FastBit on > > ubuntu. > > > > Whenever I run queries with certain columns in the where clause > I get > > the following errors. > > > > for column K100 : Process killed > > for column K10 :Segmentation Fault > > > > I did not create indexes on any of the columns, since my > understanding > > was that FastBit creates indexes as and when needed. As I ran my > > queries indexes got created. But for the columns mentioned above I > > repeatedly get the same error for all select statements with these > > columns in the where clause. > > > > I tried deleting all the other indexes to clear up some memory for > > these(K100, K10) column indexes(I think this might be happening > > because of memory shortage). It did not fix the problem. Could some > > shed some light on what might be causing this error or how to > debug or > > fix this. > > > > > > Thanks in advance, > > Deepa > > > > > > > > > > > > _______________________________________________ > > FastBit-users mailing list > > [email protected]<mailto:[email protected]> > <mailto:[email protected]<mailto:[email protected]>> > > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > >
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
