Hi Juan,
We had never tested varstrip placement with xattrs... Clearly you just
hit upon a bug that I knew was waiting to happen all this while but
did not get to fixing it... :(
Can you try the attached patch against pvfs-2.6.* and see if it works?
You still cannot control which specific I/O node a file will be
created since the
varstrip placement is with respect to some ordinal numbering of servers
which can and is typically random :)
Sample invocation of setfattr could be like
% setfattr -n "user.pvfs2.dist_params" -v "strips:0:512;1:1024"
/mnt/pvfs2/testdir/
will take bytes 0-511 on node 0, 511-1535 on node 1 and so on...
Note how fragile the parsing logic and requirement on how the distrib
params are to be constructed. Very sorry about that.. We really should
find a better way for expressing parameters I think...
If you really need very specific server location placement, you have
to checkout Julian's branch from HEAD (kunkel-hint-branch) and figure
out how to invoke PVFS_sys_create() with the hint parameters. NOTE:
You cannot use xattrs (either system or vfs) since noone has done that
yet with Julian's server-placement hint branch yet...should not be too
hard though.
let me know if this patch still does not get things working again..
thanks,
Murali
On 1/12/07, Juan Piernas Canovas <[EMAIL PROTECTED]> wrote:
Hi folks,
I'm testing PVFS2, and one of the things I want to do is to control in
which I/O node a specific file will be stored. With the "simple stripe"
distribution, I can create a directory and specify for it 1 as the
number of datafiles to be used. The problem is that every file created
in that directory is stored in a different I/O node in a round-robin
fashion.
With the varstrip distribution, you can, in theory at least, indicate
which I/O nodes, and which chunk size in every node, you want to use for
a file. But this distribution does not work for me. I have tried the
following:
$ mkdir testdir
$ setfattr -n "user.pvfs2.dist_name" -v "varstrip_dist" testdir
$ setfattr -n "user.pvfs2.dist_params" -v "strips:1:512" testdir
$ cp somefile testdir
cp: writing `testdir/somefile': Unknown error 132
I have also tried to run:
$ setfattr -n "user.pvfs2.dist_params" -v "1:512" testdir
but the cp command gets the same error.
Any clue? Is there any way to specific where I can store a file in
PVFS2?
Thanks in advance,
Juan.
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
------------------------------------------------------------------------
--- src/client/sysint/sys-create.sm.old 2007-01-12 19:32:00.776908000 -0800
+++ src/client/sysint/sys-create.sm 2007-01-12 19:44:49.307825000 -0800
@@ -15,7 +15,9 @@
#include "client-state-machine.h"
#include "pvfs2-debug.h"
+#include "pvfs2-dist-basic.h"
#include "pvfs2-dist-simple-stripe.h"
+#include "pvfs2-dist-varstrip.h"
#include "job.h"
#include "gossip.h"
#include "str-utils.h"
@@ -1134,34 +1136,50 @@
{
gossip_debug(GOSSIP_CLIENT_DEBUG, "distribution parameter %s, value
%s\n",
key[i], val[i]);
- /* NOTE: just as in server-config.c when parsing "Param" and
- * "Value" fields, we will assume that all values are 64 bit
- * integers. The only difference here is that we scan
- * directly into a 64 bit integer, rather than converting
- * from the int format that dotconf supports.
- */
- ret = sscanf(val[i], SCANF_lld, &tmp_val);
- if(ret != 1)
- {
- gossip_err("Error: unsupported type for distribution parameter %s, value %s found in directory hints.\n",
- key[i], val[i]);
- gossip_err("Error: continuing anyway.\n");
- }
- else
- {
+ if (strncmp(current_dist->dist_name,
+ PVFS_DIST_SIMPLE_STRIPE_NAME,
+ PVFS_DIST_SIMPLE_STRIPE_NAME_SIZE) == 0) {
+ /* NOTE: just as in server-config.c when parsing "Param"
and
+ * "Value" fields, we will assume that all values are 64 bit
+ * integers. The only difference here is that we scan
+ * directly into a 64 bit integer, rather than converting
+ * from the int format that dotconf supports.
+ */
+ ret = sscanf(val[i], SCANF_lld, &tmp_val);
+ if(ret != 1)
+ {
+ gossip_err("Error: unsupported type for distribution parameter %s, value %s found in directory hints.\n",
+ key[i], val[i]);
+ gossip_err("Error: continuing anyway.\n");
+ }
+ else
+ {
+
if(current_dist->methods->set_param(current_dist->dist_name,
+ current_dist->params,
+ key[i],
+ &tmp_val))
+ {
+
+ gossip_err("Error: could not override hinted
distribution parameter %s, value %s found in directory hints\n",
+ key[i],
+ val[i]);
+ }
+ }
+ } else if (strncmp(current_dist->dist_name,
+ PVFS_DIST_VARSTRIP_NAME,
+ PVFS_DIST_VARSTRIP_NAME_SIZE) == 0) {
if(current_dist->methods->set_param(current_dist->dist_name,
- current_dist->params,
- key[i],
- &tmp_val))
+ current_dist->params,
+ key[i],
+ val[i]))
{
gossip_err("Error: could not override hinted distribution parameter %s, value %s found in directory hints\n",
- key[i],
- val[i]);
+ key[i], val[i]);
}
- }
- free(key[i]);
- free(val[i]);
+ }
+ free(key[i]);
+ free(val[i]);
}
free(key);
free(val);
--- src/common/misc/str-utils.c.old 2007-01-12 19:26:49.129773000 -0800
+++ src/common/misc/str-utils.c 2007-01-12 19:27:49.448030000 -0800
@@ -688,6 +688,7 @@
* The given string must be comma separated, and each
* segment within the comma regions must be of of
* the form key:val.
+ * val itself can contain colons.
* Return the number of such keyval pairs and a
* pointer to a double dimensional array of keys and values.
* In case of errors, a -ve PVFS error is returned.
@@ -697,6 +698,7 @@
* NULL - return -PVFS_EINVAL
* ab:23 - return nkey as 1, pkey <"ab">, pval <"23">
* ab:23,bc:34 - returns nkey as 2, pkey <"ab", "bc">, pval<"23", "34">
+ * ab:2:3 - returns nkey as 1, pkey <"ab">, pval <"2:3">
*
*/
int PINT_split_keyvals(char *string, int *nkey,
@@ -780,8 +782,10 @@
ptr2 = strdup(val[i]);
if (ptr1 == NULL || ptr2 == NULL)
break;
+#if 0
if (strchr(ptr1, ':') || strchr(ptr2, ':'))
break;
+#endif
key[i] = ptr1;
val[i] = ptr2;
}
