Hmm.. it seems that situation is not good. GET Bucket (aka List Objects) of Riak CS uses a kind of coverage operation of Riak (KV), and it shows timeout. There may be some log at riak nodes. You have to look into logs in all nodes because all nodes participate in coverage operations.
Could you try some further commands? 1. for sanity check, s3cmd -c s3cfg1 ls 2. To try list objects for completely empty buckets, s3cmd -c s3cfg1 mb s3://<any-new-bucket-name-you-want> s3cmd -c s3cfg1 ls s3://<any-new-bucket-name-you-want> 3. Another coverage operation, without riak cs curl -v 'http://127.0.0.1:8098/buckets/foobar/keys?keys=true' # 127.0.0.1 and 8098 should be changed to HTTP host and port # (*NOT* PB host/port) of one of riak nodes Thanks, Shino On Thu, Aug 27, 2015 at 6:41 PM, changmao wang <wang.chang...@gmail.com> wrote: > Shunichi, > > Just now, I followed your direction to change fold_objects_for_list_keys to > true, and restart riak-cs service. > > > sed -i '/fold_objects_for_list_keys/ s/false/true/g' > /etc/riak-cs/app.config; riak-cs restart > > After that, I run below command and got same error. > > s3cmd -c s3cfg1 ls s3://stock/XSHE/0/000050/2008/XSHE-000050-20080102 -d > > ... > DEBUG: Unicodising 'ls' using UTF-8 > DEBUG: Unicodising 's3://stock/XSHE/0/000050/2008/XSHE-000050-20080102' > using UTF-8 > DEBUG: Command: ls > DEBUG: Bucket 's3://stock': > DEBUG: String 'XSHE/0/000050/2008/XSHE-000050-20080102' encoded to > 'XSHE/0/000050/2008/XSHE-000050-20080102' > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Thu, 27 Aug 2015 09:35:51 > +0000\n/stock/' > DEBUG: CreateRequest: resource[uri]=/ > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Thu, 27 Aug 2015 09:35:51 > +0000\n/stock/' > DEBUG: Processing request, please wait... > DEBUG: get_hostname(stock): stock.api2.cloud-datayes.com > DEBUG: format_uri(): > http://stock.api2.cloud-datayes.com/?prefix=XSHE/0/000050/2008/XSHE-000050-20080102&delimiter=/ > WARNING: Retrying failed request: > /?prefix=XSHE/0/000050/2008/XSHE-000050-20080102&delimiter=/ (timed out) > WARNING: Waiting 3 sec... > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Thu, 27 Aug 2015 09:37:34 > +0000\n/stock/' > DEBUG: Processing request, please wait... > DEBUG: get_hostname(stock): stock.api2.cloud-datayes.com > DEBUG: format_uri(): > http://stock.api2.cloud-datayes.com/?prefix=XSHE/0/000050/2008/XSHE-000050-20080102&delimiter=/ > > below was last 10 errors from "/var/log/riak-cs/console.log" > 2015-08-27 17:35:40.085 [error] > <0.27146.26>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user > record for s3 failed. Reason: no_user_key > 2015-08-27 17:37:34.744 [error] > <0.27147.26>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user > record for s3 failed. Reason: no_user_key > 2015-08-27 17:37:49.356 [error] > <0.27146.26>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user > record for s3 failed. Reason: no_user_key > 2015-08-27 17:39:49.249 [error] > <0.27147.26>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user > record for s3 failed. Reason: no_user_key > 2015-08-27 17:39:54.811 [error] > <0.27146.26>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user > record for s3 failed. Reason: no_user_key > > > > On Wed, Aug 26, 2015 at 5:35 PM, Shunichi Shinohara <sh...@basho.com> wrote: >> >> Sorry for delay and thanks for further information. >> The log of Riak CS shows timeout in list query to Riak: >> 2015-08-24 16:59:49.717 [error] <0.18027.7> gen_fsm <0.18027.7> in >> state >> waiting_list_keys terminated with reason: <<"timeout">> >> >> There are some kinds of possibility for timeout, e.g. hardware resource. >> But if the timeout is occured by large number of objects (manifests) in >> your >> bucket, improvement from 1.4.0 may help. >> It can be used by setting >> {fold_objects_for_list_keys, true} >> in your app.config's riak_cs section or executing >> application:set_env(riak_cs, fold_objects_for_list_keys, true). >> by attaching to riak-cs node. >> For more information about it, please refer the original PR [1]. >> >> [1] https://github.com/basho/riak_cs/pull/600 >> -- >> Shunichi Shinohara >> Basho Japan KK >> >> >> On Wed, Aug 26, 2015 at 2:04 PM, Stanislav Vlasov >> <stanislav....@gmail.com> wrote: >> > 2015-08-25 11:03 GMT+05:00 changmao wang <wang.chang...@gmail.com>: >> >> Any ideas on this issue? >> > >> > Can you check credentials with another client? >> > s3curl, for example? >> > >> > I got some bugs in s3cmd after debian upgrade, so if another client >> > works, than s3cmd has bug. >> > >> >> On Mon, Aug 24, 2015 at 5:09 PM, changmao wang >> >> <wang.chang...@gmail.com> >> >> wrote: >> >>> >> >>> Please check attached file for details. >> >>> >> >>> On Mon, Aug 24, 2015 at 4:48 PM, Shunichi Shinohara <sh...@basho.com> >> >>> wrote: >> >>>> >> >>>> Then, back to my first questions: >> >>>> Could you provide results following commands with s3cfg1? >> >>>> - s3cmd ls >> >>>> - s3cmd info s3://stock >> >>>> >> >>>> From log file, gc index queries timed out again and again. >> >>>> Not sure but it may be subtle situation... >> >>>> >> >>>> -- >> >>>> Shunichi Shinohara >> >>>> Basho Japan KK >> >>>> >> >>>> >> >>>> On Mon, Aug 24, 2015 at 11:03 AM, changmao wang >> >>>> <wang.chang...@gmail.com> >> >>>> wrote: >> >>>> > 1. root@cluster1-hd10:~# grep cs_root_host /etc/riak-cs/app.config >> >>>> > {cs_root_host, "api2.cloud-datayes.com"}, >> >>>> > root@cluster1-hd10:~# grep host_base .s3cfg >> >>>> > host_base = api2.cloud-datayes.com >> >>>> > root@cluster1-hd10:~# grep host_base s3cfg1 >> >>>> > host_base = api2.cloud-datayes.com >> >>>> > >> >>>> > 2. please check attached file for "s3cmd -d" output and >> >>>> > '/etc/riak-cs/console.log'. >> >>>> > >> >>>> > >> >>>> > On Mon, Aug 24, 2015 at 9:54 AM, Shunichi Shinohara >> >>>> > <sh...@basho.com> >> >>>> > wrote: >> >>>> >> >> >>>> >> What is "api2.cloud-datayes.com"? Your s3cfg attached at the first >> >>>> >> one >> >>>> >> in this email thread >> >>>> >> does not include it. Please make sure you provide correct / >> >>>> >> consistent >> >>>> >> information to >> >>>> >> debug the issue. >> >>>> >> >> >>>> >> - What is your riak cs config "cs_root_host"? >> >>>> >> - What is your host_base in s3cfg that you USE? >> >>>> >> - What is your host_bucket in s3cfg? >> >>>> >> >> >>>> >> Also, please attach s3cmd debug output AND riak cs console log at >> >>>> >> the >> >>>> >> same >> >>>> >> time >> >>>> >> interval. >> >>>> >> -- >> >>>> >> Shunichi Shinohara >> >>>> >> Basho Japan KK >> >>>> >> >> >>>> >> >> >>>> >> On Mon, Aug 24, 2015 at 10:42 AM, changmao wang >> >>>> >> <wang.chang...@gmail.com> >> >>>> >> wrote: >> >>>> >> > I'm not sure who created it. This's a legacy production system. >> >>>> >> > >> >>>> >> > Just now, I used another "s3cfg" file to access it. Below is my >> >>>> >> > output: >> >>>> >> > root@cluster1-hd10:~# s3cmd -c s3cfg1 info >> >>>> >> > s3://stock/XSHE/0/000050/2008/XSHE-000050-20080102 >> >>>> >> > s3://stock/XSHE/0/000050/2008/XSHE-000050-20080102 (object): >> >>>> >> > File size: 397535 >> >>>> >> > Last mod: Thu, 05 Dec 2013 03:19:00 GMT >> >>>> >> > MIME type: binary/octet-stream >> >>>> >> > MD5 sum: feb2609ecfc9bb21549f2401a5c9477d >> >>>> >> > ACL: stockwrite: FULL_CONTROL >> >>>> >> > ACL: *anon*: READ >> >>>> >> > URL: >> >>>> >> > >> >>>> >> > >> >>>> >> > http://stock.s3.amazonaws.com/XSHE/0/000050/2008/XSHE-000050-20080102 >> >>>> >> > root@cluster1-hd10:~# s3cmd -c s3cfg1 ls >> >>>> >> > s3://stock/XSHE/0/000050/2008/XSHE-000050-20080102 -d >> >>>> >> > DEBUG: ConfigParser: Reading file 's3cfg1' >> >>>> >> > DEBUG: ConfigParser: access_key->TE...17_chars...0 >> >>>> >> > DEBUG: ConfigParser: bucket_location->US >> >>>> >> > DEBUG: ConfigParser: cloudfront_host->cloudfront.amazonaws.com >> >>>> >> > DEBUG: ConfigParser: >> >>>> >> > cloudfront_resource->/2010-07-15/distribution >> >>>> >> > DEBUG: ConfigParser: default_mime_type->binary/octet-stream >> >>>> >> > DEBUG: ConfigParser: delete_removed->False >> >>>> >> > DEBUG: ConfigParser: dry_run->False >> >>>> >> > DEBUG: ConfigParser: encoding->UTF-8 >> >>>> >> > DEBUG: ConfigParser: encrypt->False >> >>>> >> > DEBUG: ConfigParser: follow_symlinks->False >> >>>> >> > DEBUG: ConfigParser: force->False >> >>>> >> > DEBUG: ConfigParser: get_continue->False >> >>>> >> > DEBUG: ConfigParser: gpg_command->/usr/bin/gpg >> >>>> >> > DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d --verbose >> >>>> >> > --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s >> >>>> >> > -o >> >>>> >> > %(output_file)s %(input_file)s >> >>>> >> > DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c --verbose >> >>>> >> > --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s >> >>>> >> > -o >> >>>> >> > %(output_file)s %(input_file)s >> >>>> >> > DEBUG: ConfigParser: gpg_passphrase->...-3_chars... >> >>>> >> > DEBUG: ConfigParser: guess_mime_type->True >> >>>> >> > DEBUG: ConfigParser: host_base->api2.cloud-datayes.com >> >>>> >> > DEBUG: ConfigParser: >> >>>> >> > host_bucket->%(bucket)s.api2.cloud-datayes.com >> >>>> >> > DEBUG: ConfigParser: human_readable_sizes->False >> >>>> >> > DEBUG: ConfigParser: list_md5->False >> >>>> >> > DEBUG: ConfigParser: log_target_prefix-> >> >>>> >> > DEBUG: ConfigParser: preserve_attrs->True >> >>>> >> > DEBUG: ConfigParser: progress_meter->True >> >>>> >> > DEBUG: ConfigParser: proxy_host->10.21.136.81 >> >>>> >> > DEBUG: ConfigParser: proxy_port->8080 >> >>>> >> > DEBUG: ConfigParser: recursive->False >> >>>> >> > DEBUG: ConfigParser: recv_chunk->4096 >> >>>> >> > DEBUG: ConfigParser: reduced_redundancy->False >> >>>> >> > DEBUG: ConfigParser: secret_key->Hk...37_chars...= >> >>>> >> > DEBUG: ConfigParser: send_chunk->4096 >> >>>> >> > DEBUG: ConfigParser: simpledb_host->sdb.amazonaws.com >> >>>> >> > DEBUG: ConfigParser: skip_existing->False >> >>>> >> > DEBUG: ConfigParser: socket_timeout->100 >> >>>> >> > DEBUG: ConfigParser: urlencoding_mode->normal >> >>>> >> > DEBUG: ConfigParser: use_https->False >> >>>> >> > DEBUG: ConfigParser: verbosity->WARNING >> >>>> >> > DEBUG: Updating Config.Config encoding -> UTF-8 >> >>>> >> > DEBUG: Updating Config.Config follow_symlinks -> False >> >>>> >> > DEBUG: Updating Config.Config verbosity -> 10 >> >>>> >> > DEBUG: Unicodising 'ls' using UTF-8 >> >>>> >> > DEBUG: Unicodising >> >>>> >> > 's3://stock/XSHE/0/000050/2008/XSHE-000050-20080102' >> >>>> >> > using UTF-8 >> >>>> >> > DEBUG: Command: ls >> >>>> >> > DEBUG: Bucket 's3://stock': >> >>>> >> > DEBUG: String 'XSHE/0/000050/2008/XSHE-000050-20080102' encoded >> >>>> >> > to >> >>>> >> > 'XSHE/0/000050/2008/XSHE-000050-20080102' >> >>>> >> > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Mon, 24 Aug 2015 >> >>>> >> > 01:36:01 >> >>>> >> > +0000\n/stock/' >> >>>> >> > DEBUG: CreateRequest: resource[uri]=/ >> >>>> >> > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Mon, 24 Aug 2015 >> >>>> >> > 01:36:01 >> >>>> >> > +0000\n/stock/' >> >>>> >> > DEBUG: Processing request, please wait... >> >>>> >> > DEBUG: get_hostname(stock): stock.api2.cloud-datayes.com >> >>>> >> > DEBUG: format_uri(): >> >>>> >> > >> >>>> >> > >> >>>> >> > >> >>>> >> > http://stock.api2.cloud-datayes.com/?prefix=XSHE/0/000050/2008/XSHE-000050-20080102&delimiter=/ >> >>>> >> > WARNING: Retrying failed request: >> >>>> >> > /?prefix=XSHE/0/000050/2008/XSHE-000050-20080102&delimiter=/ >> >>>> >> > ('') >> >>>> >> > WARNING: Waiting 3 sec... >> >>>> >> > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Mon, 24 Aug 2015 >> >>>> >> > 01:37:05 >> >>>> >> > +0000\n/stock/' >> >>>> >> > DEBUG: Processing request, please wait... >> >>>> >> > DEBUG: get_hostname(stock): stock.api2.cloud-datayes.com >> >>>> >> > DEBUG: format_uri(): >> >>>> >> > >> >>>> >> > >> >>>> >> > >> >>>> >> > http://stock.api2.cloud-datayes.com/?prefix=XSHE/0/000050/2008/XSHE-000050-20080102&delimiter=/ >> >>>> >> > WARNING: Retrying failed request: >> >>>> >> > /?prefix=XSHE/0/000050/2008/XSHE-000050-20080102&delimiter=/ >> >>>> >> > ('') >> >>>> >> > WARNING: Waiting 6 sec... >> >>>> >> > .... >> >>>> >> > >> >>>> >> > >> >>>> >> > >> >>>> >> > On Mon, Aug 24, 2015 at 9:17 AM, Shunichi Shinohara >> >>>> >> > <sh...@basho.com> >> >>>> >> > wrote: >> >>>> >> >> >> >>>> >> >> The result of "s3cmd ls" (aka, GET Service API) indicates there >> >>>> >> >> is no bucket with name "stock": >> >>>> >> >> >> >>>> >> >> > root@cluster-s3-hd1:~# s3cmd ls >> >>>> >> >> > 2013-12-01 06:45 s3://test >> >>>> >> >> >> >>>> >> >> Have you created it? >> >>>> >> >> >> >>>> >> >> -- >> >>>> >> >> Shunichi Shinohara >> >>>> >> >> Basho Japan KK >> >>>> >> >> >> >>>> >> >> >> >>>> >> >> On Mon, Aug 24, 2015 at 10:14 AM, changmao wang >> >>>> >> >> <wang.chang...@gmail.com> >> >>>> >> >> wrote: >> >>>> >> >> > Shunichi, >> >>>> >> >> > >> >>>> >> >> > Thanks for your reply. Below is my command result: >> >>>> >> >> > root@cluster-s3-hd1:~# s3cmd ls >> >>>> >> >> > 2013-12-01 06:45 s3://test >> >>>> >> >> > root@cluster-s3-hd1:~# s3cmd info s3://stock >> >>>> >> >> > ERROR: Access to bucket 'stock' was denied >> >>>> >> >> > root@cluster-s3-hd1:~# s3cmd info s3://stock -d >> >>>> >> >> > DEBUG: ConfigParser: Reading file '/root/.s3cfg' >> >>>> >> >> > DEBUG: ConfigParser: access_key->M2...17_chars...K >> >>>> >> >> > DEBUG: ConfigParser: bucket_location->US >> >>>> >> >> > DEBUG: ConfigParser: >> >>>> >> >> > cloudfront_host->cloudfront.amazonaws.com >> >>>> >> >> > DEBUG: ConfigParser: >> >>>> >> >> > cloudfront_resource->/2010-07-15/distribution >> >>>> >> >> > DEBUG: ConfigParser: default_mime_type->binary/octet-stream >> >>>> >> >> > DEBUG: ConfigParser: delete_removed->False >> >>>> >> >> > DEBUG: ConfigParser: dry_run->False >> >>>> >> >> > DEBUG: ConfigParser: encoding->UTF-8 >> >>>> >> >> > DEBUG: ConfigParser: encrypt->False >> >>>> >> >> > DEBUG: ConfigParser: follow_symlinks->False >> >>>> >> >> > DEBUG: ConfigParser: force->False >> >>>> >> >> > DEBUG: ConfigParser: get_continue->False >> >>>> >> >> > DEBUG: ConfigParser: gpg_command->/usr/bin/gpg >> >>>> >> >> > DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d >> >>>> >> >> > --verbose >> >>>> >> >> > --no-use-agent --batch --yes --passphrase-fd >> >>>> >> >> > %(passphrase_fd)s -o >> >>>> >> >> > %(output_file)s %(input_file)s >> >>>> >> >> > DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c >> >>>> >> >> > --verbose >> >>>> >> >> > --no-use-agent --batch --yes --passphrase-fd >> >>>> >> >> > %(passphrase_fd)s -o >> >>>> >> >> > %(output_file)s %(input_file)s >> >>>> >> >> > DEBUG: ConfigParser: gpg_passphrase->...-3_chars... >> >>>> >> >> > DEBUG: ConfigParser: guess_mime_type->True >> >>>> >> >> > DEBUG: ConfigParser: host_base->api2.cloud-datayes.com >> >>>> >> >> > DEBUG: ConfigParser: >> >>>> >> >> > host_bucket->%(bucket)s.api2.cloud-datayes.com >> >>>> >> >> > DEBUG: ConfigParser: human_readable_sizes->False >> >>>> >> >> > DEBUG: ConfigParser: list_md5->False >> >>>> >> >> > DEBUG: ConfigParser: log_target_prefix-> >> >>>> >> >> > DEBUG: ConfigParser: preserve_attrs->True >> >>>> >> >> > DEBUG: ConfigParser: progress_meter->True >> >>>> >> >> > DEBUG: ConfigParser: proxy_host->10.21.136.81 >> >>>> >> >> > DEBUG: ConfigParser: proxy_port->8080 >> >>>> >> >> > DEBUG: ConfigParser: recursive->False >> >>>> >> >> > DEBUG: ConfigParser: recv_chunk->4096 >> >>>> >> >> > DEBUG: ConfigParser: reduced_redundancy->False >> >>>> >> >> > DEBUG: ConfigParser: secret_key->1u...37_chars...= >> >>>> >> >> > DEBUG: ConfigParser: send_chunk->4096 >> >>>> >> >> > DEBUG: ConfigParser: simpledb_host->sdb.amazonaws.com >> >>>> >> >> > DEBUG: ConfigParser: skip_existing->False >> >>>> >> >> > DEBUG: ConfigParser: socket_timeout->10 >> >>>> >> >> > DEBUG: ConfigParser: urlencoding_mode->normal >> >>>> >> >> > DEBUG: ConfigParser: use_https->False >> >>>> >> >> > DEBUG: ConfigParser: verbosity->WARNING >> >>>> >> >> > DEBUG: Updating Config.Config encoding -> UTF-8 >> >>>> >> >> > DEBUG: Updating Config.Config follow_symlinks -> False >> >>>> >> >> > DEBUG: Updating Config.Config verbosity -> 10 >> >>>> >> >> > DEBUG: Unicodising 'info' using UTF-8 >> >>>> >> >> > DEBUG: Unicodising 's3://stock' using UTF-8 >> >>>> >> >> > DEBUG: Command: info >> >>>> >> >> > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Mon, 24 Aug 2015 >> >>>> >> >> > 01:11:09 >> >>>> >> >> > +0000\n/stock/?location' >> >>>> >> >> > DEBUG: CreateRequest: resource[uri]=/?location >> >>>> >> >> > DEBUG: SignHeaders: 'GET\n\n\n\nx-amz-date:Mon, 24 Aug 2015 >> >>>> >> >> > 01:11:09 >> >>>> >> >> > +0000\n/stock/?location' >> >>>> >> >> > DEBUG: Processing request, please wait... >> >>>> >> >> > DEBUG: get_hostname(stock): stock.api2.cloud-datayes.com >> >>>> >> >> > DEBUG: format_uri(): >> >>>> >> >> > http://stock.api2.cloud-datayes.com/?location >> >>>> >> >> > DEBUG: Response: {'status': 403, 'headers': {'date': 'Mon, 24 >> >>>> >> >> > Aug >> >>>> >> >> > 2015 >> >>>> >> >> > 01:11:09 GMT', 'content-length': '160', 'content-type': >> >>>> >> >> > 'application/xml', >> >>>> >> >> > 'server': 'Riak CS'}, 'reason': 'Forbidden', 'data': '<?xml >> >>>> >> >> > version="1.0" >> >>>> >> >> > >> >>>> >> >> > encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access >> >>>> >> >> > >> >>>> >> >> > >> >>>> >> >> > >> >>>> >> >> > >> >>>> >> >> > Denied</Message><Resource>/stock</Resource><RequestId></RequestId></Error>'} >> >>>> >> >> > DEBUG: S3Error: 403 (Forbidden) >> >>>> >> >> > DEBUG: HttpHeader: date: Mon, 24 Aug 2015 01:11:09 GMT >> >>>> >> >> > DEBUG: HttpHeader: content-length: 160 >> >>>> >> >> > DEBUG: HttpHeader: content-type: application/xml >> >>>> >> >> > DEBUG: HttpHeader: server: Riak CS >> >>>> >> >> > DEBUG: ErrorXML: Code: 'AccessDenied' >> >>>> >> >> > DEBUG: ErrorXML: Message: 'Access Denied' >> >>>> >> >> > DEBUG: ErrorXML: Resource: '/stock' >> >>>> >> >> > DEBUG: ErrorXML: RequestId: None >> >>>> >> >> > ERROR: Access to bucket 'stock' was denied >> >>>> >> >> > >> >>>> >> >> > On Mon, Aug 24, 2015 at 9:04 AM, Shunichi Shinohara >> >>>> >> >> > <sh...@basho.com> >> >>>> >> >> > wrote: >> >>>> >> >> >> >> >>>> >> >> >> The error message in console.log shows no user with >> >>>> >> >> >> access_key >> >>>> >> >> >> in >> >>>> >> >> >> your >> >>>> >> >> >> s3cfg. >> >>>> >> >> >> Could you provide resutls following commands? >> >>>> >> >> >> >> >>>> >> >> >> - s3cmd ls >> >>>> >> >> >> - s3cmd info s3://stock >> >>>> >> >> >> >> >>>> >> >> >> If error happens, debug print switch "-d" of s3cmd might >> >>>> >> >> >> help. >> >>>> >> >> >> >> >>>> >> >> >> [1] >> >>>> >> >> >> >> >>>> >> >> >> >> >>>> >> >> >> >> >>>> >> >> >> >> >>>> >> >> >> http://docs.basho.com/riakcs/latest/cookbooks/Account-Management/#Creating-a-User-Account >> >>>> >> >> >> >> >>>> >> >> >> -- >> >>>> >> >> >> Shunichi Shinohara >> >>>> >> >> >> Basho Japan KK >> >>>> >> >> >> >> >>>> >> >> >> >> >>>> >> >> >> On Fri, Aug 21, 2015 at 10:00 AM, changmao wang >> >>>> >> >> >> <wang.chang...@gmail.com> >> >>>> >> >> >> wrote: >> >>>> >> >> >> > Kazuhiro, >> >>>> >> >> >> > >> >>>> >> >> >> > Maybe that's not the key point. I'm using riak 1.4.2 and >> >>>> >> >> >> > follow >> >>>> >> >> >> > below >> >>>> >> >> >> > docs >> >>>> >> >> >> > to configure "s3cfg" file. >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > http://docs.basho.com/riakcs/1.4.2/cookbooks/configuration/Configuring-an-S3-Client/#Sample-s3cmd-Configuration-File-for-Production-Use >> >>>> >> >> >> > >> >>>> >> >> >> > There's no "signature_v2" parameter in "s3cfg". However, I >> >>>> >> >> >> > added >> >>>> >> >> >> > this >> >>>> >> >> >> > parameter to "s3cfg" and tried again with same errors. >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > On Thu, Aug 20, 2015 at 10:31 PM, Kazuhiro Suzuki >> >>>> >> >> >> > <k...@basho.com> >> >>>> >> >> >> > wrote: >> >>>> >> >> >> >> >> >>>> >> >> >> >> Hi Changmao, >> >>>> >> >> >> >> >> >>>> >> >> >> >> It seems your s3cmd config should include 2 items: >> >>>> >> >> >> >> >> >>>> >> >> >> >> signature_v2 = True >> >>>> >> >> >> >> host_base = api2.cloud-datayes.com >> >>>> >> >> >> >> >> >>>> >> >> >> >> Riak CS requires "signature_v2 = True" since Riak CS has >> >>>> >> >> >> >> not >> >>>> >> >> >> >> supported >> >>>> >> >> >> >> s3 authentication version 4 yet. >> >>>> >> >> >> >> You can find a sample configuration of s3cmd here to >> >>>> >> >> >> >> interact >> >>>> >> >> >> >> with >> >>>> >> >> >> >> Riak >> >>>> >> >> >> >> CS >> >>>> >> >> >> >> [1]. >> >>>> >> >> >> >> >> >>>> >> >> >> >> [1]: >> >>>> >> >> >> >> >> >>>> >> >> >> >> >> >>>> >> >> >> >> >> >>>> >> >> >> >> >> >>>> >> >> >> >> >> >>>> >> >> >> >> http://docs.basho.com/riakcs/2.0.1/cookbooks/configuration/Configuring-an-S3-Client/#Sample-s3cmd-Configuration-File-for-Production-Use >> >>>> >> >> >> >> >> >>>> >> >> >> >> Thanks, >> >>>> >> >> >> >> >> >>>> >> >> >> >> On Thu, Aug 20, 2015 at 7:44 PM, changmao wang >> >>>> >> >> >> >> <wang.chang...@gmail.com> >> >>>> >> >> >> >> wrote: >> >>>> >> >> >> >> > Just now, I used "admin_key" and "admin_secret" from >> >>>> >> >> >> >> > /etc/riak-cs/app.config >> >>>> >> >> >> >> > to run "s3cmd -c s3-stock ls s3://stock/XSHE/0/000600" >> >>>> >> >> >> >> > and I got the below error: >> >>>> >> >> >> >> > ERROR: Access to bucket 'stock' was denied >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > Below is abstract from "/var/log/riak-cs/console.log" >> >>>> >> >> >> >> > 2015-08-20 18:40:22.790 [debug] >> >>>> >> >> >> >> > <0.28085.18>@riak_cs_s3_auth:calculate_signature:129 >> >>>> >> >> >> >> > STS: >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > ["GET","\n",[],"\n",[],"\n","\n",[["x-amz-date",":",<<"Thu, >> >>>> >> >> >> >> > 20 >> >>>> >> >> >> >> > Aug >> >>>> >> >> >> >> > 2015 >> >>>> >> >> >> >> > 10:40:22 +0000">>,"\n"]],["/stock/",[]]] >> >>>> >> >> >> >> > 2015-08-20 18:40:32.861 [error] >> >>>> >> >> >> >> > <0.28153.18>@riak_cs_wm_common:maybe_create_user:223 >> >>>> >> >> >> >> > Retrieval >> >>>> >> >> >> >> > of >> >>>> >> >> >> >> > user >> >>>> >> >> >> >> > record for s3 failed. Reason: no_user_key >> >>>> >> >> >> >> > 2015-08-20 18:40:32.861 [debug] >> >>>> >> >> >> >> > <0.28153.18>@riak_cs_wm_common:post_authentication:452 >> >>>> >> >> >> >> > No >> >>>> >> >> >> >> > user >> >>>> >> >> >> >> > key >> >>>> >> >> >> >> > 2015-08-20 18:40:32.969 [debug] >> >>>> >> >> >> >> > <0.28189.18>@riak_cs_get_fsm:prepare:406 >> >>>> >> >> >> >> > Manifest: >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > {lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,122,47,83,90,48,48,50,49,55,53,67,78,47,50,48,49,48,95,50,48,49,48,45,48,52,45,50,52,95,229,133,172,229,143,184,231,171,160,231,168,139,239,188,136,50,48,49,48,229,185,180,52,230,156,136,239,188,137,46,80,68,70>>},[],"2013-12-16T23:01:12.000Z",<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,255387,<<"application/pdf">>,<<55,41,141,170,187,226,47,223,183,95,105,129,155,154,210,202>>,active,{1387,234872,598819},{1387,234872,918555},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,234872,598546}},[],undefined} >> >>>> >> >> >> >> > 2015-08-20 18:40:33.043 [debug] >> >>>> >> >> >> >> > <0.28189.18>@riak_cs_lfs_utils:range_blocks:118 >> >>>> >> >> >> >> > InitialBlock: >> >>>> >> >> >> >> > 0, >> >>>> >> >> >> >> > FinalBlock: >> >>>> >> >> >> >> > 0 >> >>>> >> >> >> >> > 2015-08-20 18:40:33.043 [debug] >> >>>> >> >> >> >> > <0.28189.18>@riak_cs_lfs_utils:range_blocks:120 >> >>>> >> >> >> >> > SkipInitial: 0, >> >>>> >> >> >> >> > KeepFinal: >> >>>> >> >> >> >> > 255387 >> >>>> >> >> >> >> > 2015-08-20 18:40:33.050 [debug] >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > <0.28189.18>@riak_cs_get_fsm:waiting_continue_or_stop:229 >> >>>> >> >> >> >> > Block >> >>>> >> >> >> >> > Servers: >> >>>> >> >> >> >> > [<0.28191.18>] >> >>>> >> >> >> >> > 2015-08-20 18:40:33.079 [debug] >> >>>> >> >> >> >> > <0.28189.18>@riak_cs_get_fsm:waiting_chunks:307 >> >>>> >> >> >> >> > Retrieved >> >>>> >> >> >> >> > block >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > {<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0} >> >>>> >> >> >> >> > 2015-08-20 18:40:33.079 [debug] >> >>>> >> >> >> >> > <0.28189.18>@riak_cs_get_fsm:perhaps_send_to_user:280 >> >>>> >> >> >> >> > Returning >> >>>> >> >> >> >> > block >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > {<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0} >> >>>> >> >> >> >> > to >> >>>> >> >> >> >> > client >> >>>> >> >> >> >> > 2015-08-20 18:40:38.218 [error] >> >>>> >> >> >> >> > <0.28086.18>@riak_cs_wm_common:maybe_create_user:223 >> >>>> >> >> >> >> > Retrieval >> >>>> >> >> >> >> > of >> >>>> >> >> >> >> > user >> >>>> >> >> >> >> > record for s3 failed. Reason: no_user_key >> >>>> >> >> >> >> > 2015-08-20 18:40:38.218 [debug] >> >>>> >> >> >> >> > <0.28086.18>@riak_cs_wm_common:post_authentication:452 >> >>>> >> >> >> >> > No >> >>>> >> >> >> >> > user >> >>>> >> >> >> >> > key >> >>>> >> >> >> >> > 2015-08-20 18:40:38.226 [debug] >> >>>> >> >> >> >> > <0.28210.18>@riak_cs_get_fsm:prepare:406 >> >>>> >> >> >> >> > Manifest: >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > {lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,104,47,83,72,54,48,48,55,53,48,67,78,47,50,48,48,55,95,50,48,48,55,45,49,49,45,50,49,95,230,177,159,228,184,173,232,141,175,228,184,154,229,133,179,228,186,142,73,66,69,95,53,232,141,175,229,147,129,232,142,183,229,190,151,228,186,140,230,156,159,228,184,180,229,186,138,230,137,185,230,150,135,231,154,132,229,133,172,229,145,138,229,143,138,233,163,142,233,153,169,230,143,144,231,164,186,46,112,100,102>>},[],"2013-12-15T09:04:48.000Z",<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,89863,<<"application/pdf">>,<<139,151,203,173,6,111,222,48,17,81,102,170,216,66,193,77>>,active,{1387,98288,545827},{1387,98288,618409},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","e f38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,98288,545618}},[],undefined} >> >>>> >> >> >> >> > 2015-08-20 18:40:38.280 [debug] >> >>>> >> >> >> >> > <0.28210.18>@riak_cs_lfs_utils:range_blocks:118 >> >>>> >> >> >> >> > InitialBlock: >> >>>> >> >> >> >> > 0, >> >>>> >> >> >> >> > FinalBlock: >> >>>> >> >> >> >> > 0 >> >>>> >> >> >> >> > 2015-08-20 18:40:38.280 [debug] >> >>>> >> >> >> >> > <0.28210.18>@riak_cs_lfs_utils:range_blocks:120 >> >>>> >> >> >> >> > SkipInitial: 0, >> >>>> >> >> >> >> > KeepFinal: >> >>>> >> >> >> >> > 89863 >> >>>> >> >> >> >> > 2015-08-20 18:40:38.280 [debug] >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > <0.28210.18>@riak_cs_get_fsm:waiting_continue_or_stop:229 >> >>>> >> >> >> >> > Block >> >>>> >> >> >> >> > Servers: >> >>>> >> >> >> >> > [<0.28212.18>] >> >>>> >> >> >> >> > 2015-08-20 18:40:38.343 [debug] >> >>>> >> >> >> >> > <0.28210.18>@riak_cs_get_fsm:waiting_chunks:307 >> >>>> >> >> >> >> > Retrieved >> >>>> >> >> >> >> > block >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > {<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,0} >> >>>> >> >> >> >> > 2015-08-20 18:40:38.344 [debug] >> >>>> >> >> >> >> > <0.28210.18>@riak_cs_get_fsm:perhaps_send_to_user:280 >> >>>> >> >> >> >> > Returning >> >>>> >> >> >> >> > block >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > {<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,0} >> >>>> >> >> >> >> > to >> >>>> >> >> >> >> > client >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > On Thu, Aug 20, 2015 at 6:04 PM, Stanislav Vlasov >> >>>> >> >> >> >> > <stanislav....@gmail.com> >> >>>> >> >> >> >> > wrote: >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> 2015-08-20 14:47 GMT+05:00 changmao wang >> >>>> >> >> >> >> >> <wang.chang...@gmail.com>: >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> > what's your meaning of domain name of >> >>>> >> >> >> >> >> > /etc/riak-cs/app.config >> >>>> >> >> >> >> >> > and >> >>>> >> >> >> >> >> > ~/.s3cfg? >> >>>> >> >> >> >> >> > I guess it's cs_root_host parameter from >> >>>> >> >> >> >> >> > /etc/riak-cs/app.config >> >>>> >> >> >> >> >> > and >> >>>> >> >> >> >> >> > host_base from '~/.s3cfg'. >> >>>> >> >> >> >> >> > If so, there're same as "api2.cloud-datayes.com". >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> Yes, is that i mean, but i see, it is not your case >> >>>> >> >> >> >> >> Try to set {level, debug} in lager_file_backend >> >>>> >> >> >> >> >> section >> >>>> >> >> >> >> >> for >> >>>> >> >> >> >> >> console.log. >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> > However, I can not ping this host from localhost. >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> It's ok, if you write proper proxy_host and proxy_port >> >>>> >> >> >> >> >> in >> >>>> >> >> >> >> >> .s3cfg >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> > On Thu, Aug 20, 2015 at 5:23 PM, Stanislav Vlasov >> >>>> >> >> >> >> >> > <stanislav....@gmail.com> >> >>>> >> >> >> >> >> > wrote: >> >>>> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> 2015-08-20 13:57 GMT+05:00 changmao wang >> >>>> >> >> >> >> >> >> <wang.chang...@gmail.com>: >> >>>> >> >> >> >> >> >> > somebody watching on this? >> >>>> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> Do you set up same domain in riak-cs.conf and in >> >>>> >> >> >> >> >> >> .s3cfg? >> >>>> >> >> >> >> >> >> I got such error in this case. >> >>>> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> > On Wed, Aug 19, 2015 at 9:01 AM, changmao wang >> >>>> >> >> >> >> >> >> > <wang.chang...@gmail.com> >> >>>> >> >> >> >> >> >> > wrote: >> >>>> >> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> Matthew, >> >>>> >> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> I used s3cmd --configure to generate ".s3cfg" >> >>>> >> >> >> >> >> >> >> config >> >>>> >> >> >> >> >> >> >> file >> >>>> >> >> >> >> >> >> >> and >> >>>> >> >> >> >> >> >> >> then >> >>>> >> >> >> >> >> >> >> access >> >>>> >> >> >> >> >> >> >> RIAK service by s3cmd. >> >>>> >> >> >> >> >> >> >> The access_key and secret_key from ".s3cfg" is >> >>>> >> >> >> >> >> >> >> same >> >>>> >> >> >> >> >> >> >> as >> >>>> >> >> >> >> >> >> >> admin_key >> >>>> >> >> >> >> >> >> >> and >> >>>> >> >> >> >> >> >> >> admin_secret from "/etc/riak-cs/app.config". >> >>>> >> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> However, I got error as below using s3cmd to >> >>>> >> >> >> >> >> >> >> access >> >>>> >> >> >> >> >> >> >> one >> >>>> >> >> >> >> >> >> >> bucket. >> >>>> >> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> root@cluster-s3-hd1:~# s3cmd -c /root/.s3cfg ls >> >>>> >> >> >> >> >> >> >> s3://pipeline/article/111.pdf >> >>>> >> >> >> >> >> >> >> ERROR: Access to bucket 'pipeline' was denied >> >>>> >> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> By the way, I used Riak and Riak-CS 1.4.2 on >> >>>> >> >> >> >> >> >> >> Ubuntu. >> >>>> >> >> >> >> >> >> >> Current >> >>>> >> >> >> >> >> >> >> production >> >>>> >> >> >> >> >> >> >> cluster is a legacy system without documents for >> >>>> >> >> >> >> >> >> >> co-workers. >> >>>> >> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> Attached file is "s3cfg" generated by "s3cmd >> >>>> >> >> >> >> >> >> >> --configure". >> >>>> >> >> >> >> >> >> >> -- >> >>>> >> >> >> >> >> >> >> Amao Wang >> >>>> >> >> >> >> >> >> >> Best & Regards >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > -- >> >>>> >> >> >> >> >> >> > Amao Wang >> >>>> >> >> >> >> >> >> > Best & Regards >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > _______________________________________________ >> >>>> >> >> >> >> >> >> > riak-users mailing list >> >>>> >> >> >> >> >> >> > riak-users@lists.basho.com >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >>>> >> >> >> >> >> >> > >> >>>> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >> >>>> >> >> >> >> >> >> -- >> >>>> >> >> >> >> >> >> Stanislav >> >>>> >> >> >> >> >> > >> >>>> >> >> >> >> >> > >> >>>> >> >> >> >> >> > >> >>>> >> >> >> >> >> > >> >>>> >> >> >> >> >> > -- >> >>>> >> >> >> >> >> > Amao Wang >> >>>> >> >> >> >> >> > Best & Regards >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> >> >>>> >> >> >> >> >> -- >> >>>> >> >> >> >> >> Stanislav >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > -- >> >>>> >> >> >> >> > Amao Wang >> >>>> >> >> >> >> > Best & Regards >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > _______________________________________________ >> >>>> >> >> >> >> > riak-users mailing list >> >>>> >> >> >> >> > riak-users@lists.basho.com >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > >> >>>> >> >> >> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >>>> >> >> >> >> > >> >>>> >> >> >> >> >> >>>> >> >> >> >> >> >>>> >> >> >> >> >> >>>> >> >> >> >> -- >> >>>> >> >> >> >> Kazuhiro Suzuki | Basho Japan KK >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > -- >> >>>> >> >> >> > Amao Wang >> >>>> >> >> >> > Best & Regards >> >>>> >> >> >> > >> >>>> >> >> >> > _______________________________________________ >> >>>> >> >> >> > riak-users mailing list >> >>>> >> >> >> > riak-users@lists.basho.com >> >>>> >> >> >> > >> >>>> >> >> >> > >> >>>> >> >> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >>>> >> >> >> > >> >>>> >> >> > >> >>>> >> >> > >> >>>> >> >> > >> >>>> >> >> > >> >>>> >> >> > -- >> >>>> >> >> > Amao Wang >> >>>> >> >> > Best & Regards >> >>>> >> > >> >>>> >> > >> >>>> >> > >> >>>> >> > >> >>>> >> > -- >> >>>> >> > Amao Wang >> >>>> >> > Best & Regards >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > -- >> >>>> > Amao Wang >> >>>> > Best & Regards >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Amao Wang >> >>> Best & Regards >> >> >> >> >> >> >> >> >> >> -- >> >> Amao Wang >> >> Best & Regards >> >> >> >> _______________________________________________ >> >> riak-users mailing list >> >> riak-users@lists.basho.com >> >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> > >> > >> > >> > -- >> > Stanislav >> > >> > _______________________________________________ >> > riak-users mailing list >> > riak-users@lists.basho.com >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > -- > Amao Wang > Best & Regards _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
