Re: [Pvfs2-developers] problems creating storage space with -f option

2006-04-26 Thread Phil Carns

Hi Murali,

I'm going to be out of town for the next few days, but I will give your 
patch a shot early next week.  Thanks for working on this!


-Phil

Murali Vilayannur wrote:

Hey guys,
Does this patch fix all the -f and -r problems that we have been having?
It seems to work fine for me on my machine...
Patch also fixes a string parsing bug in dbpf_mkpath() which fails to
create a directory whose path is /opt/pvfs2/1
where the last component is a single character...
Let me know,
Thanks,
Murali

On Wed, 26 Apr 2006, Phil Carns wrote:



After poking at this a little, it looks like the segfault is caused by
the sto_env field not being initialized in the dbpf_storage structure.
I guess I happen to be lucky enough to get the right kind of garbage in
that field to cause a segfault later.

I think the fix is to do a memset after malloc'ing the memory for sto_p
in dbpf_storage_lookup().

-Phil

Phil Carns wrote:


The code in CVS head is giving me a couple of problems when creating a
storage space:

# ./pvfs2-server simple.conf server.conf-localhost -f
[D 04/26 15:15] PVFS2 Server version 1.4.1pre1-2006-04-26-112404 starting.
pvfs2-trove-dbpf:
//tmp/pvfs2-test-space/3d8ae7af/collection_attributes.db: No such file
or directory
pvfs2-trove-dbpf:
//tmp/pvfs2-test-space/3d8ae7af/dataspace_attributes.db: No such file or
directory
pvfs2-trove-dbpf: //tmp/pvfs2-test-space/3d8ae7af/keyval.db: No such
file or directory
Segmentation fault

Despite this, the storage space seems to be created just fine.  There
seems to really be 2 problems:

1) The "no such file or directory" messages: I suspect that these error
messages might just be something being too chatty when it checks to see
if the dbs exist or not, but it makes it look like something has gone
wrong while creating the storage space.

2) The segmentation fault:  I don't see this every time, but I'm not
sure what is making the difference.  I did manage to catch it in gdb once:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210513728 (LWP 31619)]
0x08062de3 in dbpf_putdb_env (sto_path=0x80e8a70 "/tmp/pvfs2-test-space")
   at dbpf-mgmt.c:96
96  ret = my_storage_p->sto_env->close(my_storage_p->sto_env, 0);

-Phil

___
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers


___
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers










Index: src/common/misc/mkspace.c
===
RCS file: /anoncvs/pvfs2/src/common/misc/mkspace.c,v
retrieving revision 1.35
diff -u -r1.35 mkspace.c
--- src/common/misc/mkspace.c   4 Apr 2006 04:47:10 -   1.35
+++ src/common/misc/mkspace.c   26 Apr 2006 21:48:02 -
@@ -719,21 +719,18 @@
 
 if (!remove_collection_only)

 {
+ /*
+ * it is a bit weird to do a trove_finaliz() prior to blowing 
away
+ * the storage space, but this allows the __db files of the DB 
env
+ * to be blown away for the rmdir() to work correctly!
+ */
+ trove_finalize();
 ret = trove_storage_remove(storage_space, NULL, &op_id);
 mkspace_print(
 verbose, "PVFS2 Storage Space %s removed %s\n",
 storage_space, (((ret == 1) || (ret == -TROVE_ENOENT)) ?
 "successfully" : "with errors"));
 
-/*

-  we should be doing a trove finalize here, but for now we
-  can't because it will fail horribly during the sync/close
-  calls to files that we've just removed.
- 
-  an extra flag to finalize, or a static var in the dbpf-mgmt

-  methods could resolve this.
-*/
-/* trove_finalize(); */
 trove_is_initialized = 0;
 }
 return ret;
Index: src/io/trove/trove-dbpf/dbpf-keyval.c
===
RCS file: /anoncvs/pvfs2/src/io/trove/trove-dbpf/dbpf-keyval.c,v
retrieving revision 1.61
diff -u -r1.61 dbpf-keyval.c
--- src/io/trove/trove-dbpf/dbpf-keyval.c   25 Apr 2006 19:48:55 -  
1.61
+++ src/io/trove/trove-dbpf/dbpf-keyval.c   26 Apr 2006 21:48:02 -
@@ -847,7 +847,6 @@
 static int dbpf_keyval_read_list_op_svc(struct dbpf_op *op_p)
 {
 int ret = -TROVE_EINVAL, i = 0;
-int key_sz;
 struct dbpf_keyval_db_entry key_entry;
 DBT key, data;
 
Index: src/io/trove/trove-dbpf/dbpf-mgmt.c

===
RCS file: /anoncvs/pvfs2/src/io/trove/trove-dbpf/dbpf-mgmt.c,v
retrieving revision 1.70
diff -u -r1.70 dbpf-mgmt.c
--- src/io/trove/trove-dbpf/dbpf-mgmt.c 24 Apr 2006 20:10

Re: [Pvfs2-developers] problems creating storage space with -f option

2006-04-26 Thread Murali Vilayannur
Hey guys,
Does this patch fix all the -f and -r problems that we have been having?
It seems to work fine for me on my machine...
Patch also fixes a string parsing bug in dbpf_mkpath() which fails to
create a directory whose path is /opt/pvfs2/1
where the last component is a single character...
Let me know,
Thanks,
Murali

On Wed, 26 Apr 2006, Phil Carns wrote:

> After poking at this a little, it looks like the segfault is caused by
> the sto_env field not being initialized in the dbpf_storage structure.
> I guess I happen to be lucky enough to get the right kind of garbage in
> that field to cause a segfault later.
>
> I think the fix is to do a memset after malloc'ing the memory for sto_p
> in dbpf_storage_lookup().
>
> -Phil
>
> Phil Carns wrote:
> > The code in CVS head is giving me a couple of problems when creating a
> > storage space:
> >
> > # ./pvfs2-server simple.conf server.conf-localhost -f
> > [D 04/26 15:15] PVFS2 Server version 1.4.1pre1-2006-04-26-112404 starting.
> > pvfs2-trove-dbpf:
> > //tmp/pvfs2-test-space/3d8ae7af/collection_attributes.db: No such file
> > or directory
> > pvfs2-trove-dbpf:
> > //tmp/pvfs2-test-space/3d8ae7af/dataspace_attributes.db: No such file or
> > directory
> > pvfs2-trove-dbpf: //tmp/pvfs2-test-space/3d8ae7af/keyval.db: No such
> > file or directory
> > Segmentation fault
> >
> > Despite this, the storage space seems to be created just fine.  There
> > seems to really be 2 problems:
> >
> > 1) The "no such file or directory" messages: I suspect that these error
> > messages might just be something being too chatty when it checks to see
> > if the dbs exist or not, but it makes it look like something has gone
> > wrong while creating the storage space.
> >
> > 2) The segmentation fault:  I don't see this every time, but I'm not
> > sure what is making the difference.  I did manage to catch it in gdb once:
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread -1210513728 (LWP 31619)]
> > 0x08062de3 in dbpf_putdb_env (sto_path=0x80e8a70 "/tmp/pvfs2-test-space")
> > at dbpf-mgmt.c:96
> > 96  ret = my_storage_p->sto_env->close(my_storage_p->sto_env, 0);
> >
> > -Phil
> >
> > ___
> > Pvfs2-developers mailing list
> > Pvfs2-developers@beowulf-underground.org
> > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
> ___
> Pvfs2-developers mailing list
> Pvfs2-developers@beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
>Index: src/common/misc/mkspace.c
===
RCS file: /anoncvs/pvfs2/src/common/misc/mkspace.c,v
retrieving revision 1.35
diff -u -r1.35 mkspace.c
--- src/common/misc/mkspace.c   4 Apr 2006 04:47:10 -   1.35
+++ src/common/misc/mkspace.c   26 Apr 2006 21:48:02 -
@@ -719,21 +719,18 @@
 
 if (!remove_collection_only)
 {
+ /*
+ * it is a bit weird to do a trove_finaliz() prior to blowing 
away
+ * the storage space, but this allows the __db files of the DB 
env
+ * to be blown away for the rmdir() to work correctly!
+ */
+ trove_finalize();
 ret = trove_storage_remove(storage_space, NULL, &op_id);
 mkspace_print(
 verbose, "PVFS2 Storage Space %s removed %s\n",
 storage_space, (((ret == 1) || (ret == -TROVE_ENOENT)) ?
 "successfully" : "with errors"));
 
-/*
-  we should be doing a trove finalize here, but for now we
-  can't because it will fail horribly during the sync/close
-  calls to files that we've just removed.
- 
-  an extra flag to finalize, or a static var in the dbpf-mgmt
-  methods could resolve this.
-*/
-/* trove_finalize(); */
 trove_is_initialized = 0;
 }
 return ret;
Index: src/io/trove/trove-dbpf/dbpf-keyval.c
===
RCS file: /anoncvs/pvfs2/src/io/trove/trove-dbpf/dbpf-keyval.c,v
retrieving revision 1.61
diff -u -r1.61 dbpf-keyval.c
--- src/io/trove/trove-dbpf/dbpf-keyval.c   25 Apr 2006 19:48:55 -  
1.61
+++ src/io/trove/trove-dbpf/dbpf-keyval.c   26 Apr 2006 21:48:02 -
@@ -847,7 +847,6 @@
 static int dbpf_keyval_read_list_op_svc(struct dbpf_op *op_p)
 {
 int ret = -TROVE_EINVAL, i = 0;
-int key_sz;
 struct dbpf_keyval_db_entry key_entry;
 DBT key, data;
 
Index: src/io/trove/trove-dbpf/dbpf-mgmt.c
===
RCS file: /anoncvs/pvfs2/src/io/trove/trove-dbpf/dbpf-mgmt.c,v
retrieving revision 1.70
diff -u -r1.70 dbpf-mgmt.c
--- src/io/trove/trove-dbpf/dbpf-mgmt.c 24 Apr 2006 20:10:40 -  1.70
+++ src/io/trove/trove-dbpf/dbpf-mgmt.c 26 Apr 2006 21:48:03 -
@@ -48,

Re: [Pvfs2-developers] problems creating storage space with -f option

2006-04-26 Thread Murali Vilayannur
Hi Phil,
Indeed. I forgot to commit this and another fix for the -r option to
remove the storage space..
I will do them right away..
Thanks for pointing it out!
Murali

On Wed, 26 Apr 2006, Phil Carns wrote:

> After poking at this a little, it looks like the segfault is caused by
> the sto_env field not being initialized in the dbpf_storage structure.
> I guess I happen to be lucky enough to get the right kind of garbage in
> that field to cause a segfault later.
>
> I think the fix is to do a memset after malloc'ing the memory for sto_p
> in dbpf_storage_lookup().
>
> -Phil
>
> Phil Carns wrote:
> > The code in CVS head is giving me a couple of problems when creating a
> > storage space:
> >
> > # ./pvfs2-server simple.conf server.conf-localhost -f
> > [D 04/26 15:15] PVFS2 Server version 1.4.1pre1-2006-04-26-112404 starting.
> > pvfs2-trove-dbpf:
> > //tmp/pvfs2-test-space/3d8ae7af/collection_attributes.db: No such file
> > or directory
> > pvfs2-trove-dbpf:
> > //tmp/pvfs2-test-space/3d8ae7af/dataspace_attributes.db: No such file or
> > directory
> > pvfs2-trove-dbpf: //tmp/pvfs2-test-space/3d8ae7af/keyval.db: No such
> > file or directory
> > Segmentation fault
> >
> > Despite this, the storage space seems to be created just fine.  There
> > seems to really be 2 problems:
> >
> > 1) The "no such file or directory" messages: I suspect that these error
> > messages might just be something being too chatty when it checks to see
> > if the dbs exist or not, but it makes it look like something has gone
> > wrong while creating the storage space.
> >
> > 2) The segmentation fault:  I don't see this every time, but I'm not
> > sure what is making the difference.  I did manage to catch it in gdb once:
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread -1210513728 (LWP 31619)]
> > 0x08062de3 in dbpf_putdb_env (sto_path=0x80e8a70 "/tmp/pvfs2-test-space")
> > at dbpf-mgmt.c:96
> > 96  ret = my_storage_p->sto_env->close(my_storage_p->sto_env, 0);
> >
> > -Phil
> >
> > ___
> > Pvfs2-developers mailing list
> > Pvfs2-developers@beowulf-underground.org
> > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
> ___
> Pvfs2-developers mailing list
> Pvfs2-developers@beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
>
___
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers


Re: [Pvfs2-developers] problems creating storage space with -f option

2006-04-26 Thread Phil Carns
After poking at this a little, it looks like the segfault is caused by 
the sto_env field not being initialized in the dbpf_storage structure. 
I guess I happen to be lucky enough to get the right kind of garbage in 
that field to cause a segfault later.


I think the fix is to do a memset after malloc'ing the memory for sto_p 
in dbpf_storage_lookup().


-Phil

Phil Carns wrote:
The code in CVS head is giving me a couple of problems when creating a 
storage space:


# ./pvfs2-server simple.conf server.conf-localhost -f
[D 04/26 15:15] PVFS2 Server version 1.4.1pre1-2006-04-26-112404 starting.
pvfs2-trove-dbpf: 
//tmp/pvfs2-test-space/3d8ae7af/collection_attributes.db: No such file 
or directory
pvfs2-trove-dbpf: 
//tmp/pvfs2-test-space/3d8ae7af/dataspace_attributes.db: No such file or 
directory
pvfs2-trove-dbpf: //tmp/pvfs2-test-space/3d8ae7af/keyval.db: No such 
file or directory

Segmentation fault

Despite this, the storage space seems to be created just fine.  There 
seems to really be 2 problems:


1) The "no such file or directory" messages: I suspect that these error 
messages might just be something being too chatty when it checks to see 
if the dbs exist or not, but it makes it look like something has gone 
wrong while creating the storage space.


2) The segmentation fault:  I don't see this every time, but I'm not 
sure what is making the difference.  I did manage to catch it in gdb once:


Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210513728 (LWP 31619)]
0x08062de3 in dbpf_putdb_env (sto_path=0x80e8a70 "/tmp/pvfs2-test-space")
at dbpf-mgmt.c:96
96  ret = my_storage_p->sto_env->close(my_storage_p->sto_env, 0);

-Phil

___
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers


___
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers


[Pvfs2-developers] problems creating storage space with -f option

2006-04-26 Thread Phil Carns
The code in CVS head is giving me a couple of problems when creating a 
storage space:


# ./pvfs2-server simple.conf server.conf-localhost -f
[D 04/26 15:15] PVFS2 Server version 1.4.1pre1-2006-04-26-112404 starting.
pvfs2-trove-dbpf: 
//tmp/pvfs2-test-space/3d8ae7af/collection_attributes.db: No such file 
or directory
pvfs2-trove-dbpf: 
//tmp/pvfs2-test-space/3d8ae7af/dataspace_attributes.db: No such file or 
directory
pvfs2-trove-dbpf: //tmp/pvfs2-test-space/3d8ae7af/keyval.db: No such 
file or directory

Segmentation fault

Despite this, the storage space seems to be created just fine.  There 
seems to really be 2 problems:


1) The "no such file or directory" messages: I suspect that these error 
messages might just be something being too chatty when it checks to see 
if the dbs exist or not, but it makes it look like something has gone 
wrong while creating the storage space.


2) The segmentation fault:  I don't see this every time, but I'm not 
sure what is making the difference.  I did manage to catch it in gdb once:


Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210513728 (LWP 31619)]
0x08062de3 in dbpf_putdb_env (sto_path=0x80e8a70 "/tmp/pvfs2-test-space")
at dbpf-mgmt.c:96
96  ret = my_storage_p->sto_env->close(my_storage_p->sto_env, 0);

-Phil

___
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers