Re: Need help with Riak-KV (2.1.4) certificate based authentication using Java client

[Luke Bakken]

Kyle -

Verify return code: 19 (self signed certificate in certificate chain)

Since your server cert is self-signed, there's not much more that can
be done at this point I believe. My security tests use a dedicated CA
where the Root cert is available for validation
(https://github.com/basho/riak-client-tools/tree/master/test-ca)

--
Luke Bakken
Engineer
lbak...@basho.com

On Wed, Aug 31, 2016 at 3:11 PM, Nguyen, Kyle  wrote:
> Hi Luke,
>
> I am getting the following information:
>
> Verify return code: 19 (self signed certificate in certificate chain)

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

RE: Need help with Riak-KV (2.1.4) certificate based authentication using Java client

[Nguyen, Kyle]

Hi Luke,

I am getting the following information:

osboxes@osboxes:/etc/riak$ openssl s_client -debug -connect 10.0.2.15:8088
CONNECTED(0003)
write to 0x24244c0 [0x2424a60] (295 bytes => 295 (0x127))
 - 16 03 01 01 22 01 00 01-1e 03 03 d8 cb 68 b8 45   "h.E
0010 - 9a c3 54 21 86 7e 1a f5-de 11 08 41 74 6c d2 0e   ..T!.~.Atl..
0020 - 00 47 29 ec ba 05 40 f4-99 cd 0a 00 00 88 c0 30   .G)...@0
0030 - c0 2c c0 28 c0 24 c0 14-c0 0a 00 a3 00 9f 00 6b   .,.(.$.k
0040 - 00 6a 00 39 00 38 00 88-00 87 c0 32 c0 2e c0 2a   .j.9.8.2...*
0050 - c0 26 c0 0f c0 05 00 9d-00 3d 00 35 00 84 c0 12   .&...=.5
0060 - c0 08 00 16 00 13 c0 0d-c0 03 00 0a c0 2f c0 2b   ./.+
0070 - c0 27 c0 23 c0 13 c0 09-00 a2 00 9e 00 67 00 40   .'.#.g.@
0080 - 00 33 00 32 00 9a 00 99-00 45 00 44 c0 31 c0 2d   .3.2.E.D.1.-
0090 - c0 29 c0 25 c0 0e c0 04-00 9c 00 3c 00 2f 00 96   .).%...<./..
00a0 - 00 41 c0 11 c0 07 c0 0c-c0 02 00 05 00 04 00 15   .A..
00b0 - 00 12 00 09 00 ff 01 00-00 6d 00 0b 00 04 03 00   .m..
00c0 - 01 02 00 0a 00 34 00 32-00 0e 00 0d 00 19 00 0b   .4.2
00d0 - 00 0c 00 18 00 09 00 0a-00 16 00 17 00 08 00 06   
00e0 - 00 07 00 14 00 15 00 04-00 05 00 12 00 13 00 01   
00f0 - 00 02 00 03 00 0f 00 10-00 11 00 23 00 00 00 0d   ...#
0100 - 00 20 00 1e 06 01 06 02-06 03 05 01 05 02 05 03   . ..
0110 - 04 01 04 02 04 03 03 01-03 02 03 03 02 01 02 02   
0120 - 02 03 00 0f 00 01 01  ...
read from 0x24244c0 [0x2429fc0] (7 bytes => 7 (0x7))
 - 16 03 03 00 4a 02 J.
0007 - 
read from 0x24244c0 [0x2429fca] (72 bytes => 72 (0x48))
 - 00 46 03 03 57 c7 55 ad-6b 0b b8 4a 67 80 b8 f1   .F..W.U.k..Jg...
0010 - 82 1d 62 43 b0 b3 91 7b-b1 4b 51 8d 89 e2 a2 5d   ..bC...{.KQ]
0020 - dc 47 81 8e 20 d5 56 e6-a1 91 0d 37 55 58 b3 fd   .G.. .V7UX..
0030 - a7 a6 9a 16 e6 59 07 33-6f 84 b0 02 dd 6a b2 bd   .Y.3oj..
0040 - 47 ca d2 da 3a 00 3c  G...:.<
0048 - 
read from 0x24244c0 [0x2429fc3] (5 bytes => 5 (0x5))
 - 16 03 03 07 18.
read from 0x24244c0 [0x2429fc8] (1816 bytes => 1816 (0x718))
 - 0b 00 07 14 00 07 11 00-03 20 30 82 03 1c 30 82   . 0...0.
0010 - 02 04 a0 03 02 01 02 02-02 13 77 30 0d 06 09 2a   ..w0...*
0020 - 86 48 86 f7 0d 01 01 0b-05 00 30 5a 31 0b 30 09   .H0Z1.0.
0030 - 06 03 55 04 06 13 02 55-53 31 13 30 11 06 03 55   ..UUS1.0...U
0040 - 04 08 13 0a 57 61 73 68-69 6e 67 74 6f 6e 31 10   Washington1.
0050 - 30 0e 06 03 55 04 07 13-07 53 65 61 74 74 6c 65   0...USeattle
0060 - 31 24 30 22 06 03 55 04-0a 13 1b 67 65 74 61 43   1$0"..UgetaC
0070 - 65 72 74 20 2d 20 77 77-77 2e 67 65 74 61 63 65   ert - www.getace
0080 - 72 74 2e 63 6f 6d 30 1e-17 0d 31 36 30 38 32 32   rt.com0...160822
0090 - 32 32 34 32 34 30 5a 17-0d 31 36 31 30 32 31 32   224240Z..1610212
00a0 - 32 34 32 34 30 5a 30 19-31 17 30 15 06 03 55 04   24240Z0.1.0...U.
00b0 - 03 0c 0e 72 69 61 6b 40-31 32 37 2e 30 2e 30 2e   ...riak@127.0.0.
00c0 - 31 30 82 01 22 30 0d 06-09 2a 86 48 86 f7 0d 01   10.."0...*.H
00d0 - 01 01 05 00 03 82 01 0f-00 30 82 01 0a 02 82 01   .0..
00e0 - 01 00 89 61 17 00 0d 7d-de 42 bf f6 35 71 ef bf   ...a...}.B..5q..
00f0 - ef de 0a d7 6b 3a 30 69-80 f8 a1 47 3b f1 29 3a   k:0i...G;.):
0100 - 09 f0 1c f2 46 91 97 2a-c8 ca f6 64 fb 2c fc c6   F..*...d.,..
0110 - 69 d9 5a 12 ff 51 59 44-b5 89 17 45 38 c9 dd cf   i.Z..QYD...E8...
0120 - 11 08 21 77 df 64 d6 e4-2e 0b 54 97 dd 87 e9 74   ..!w.dTt
0130 - c9 73 9f 07 87 6c 9b 37-c9 f5 2e bb 3c c4 fa a7   .s...l.7<...
0140 - 01 88 a2 3f f8 92 18 5b-1e e2 ca 8e e0 b0 a3 c8   ...?...[
0150 - 8f 8d 90 6c da 6d 5a 83-1c 07 5c 50 40 67 1f 4d   ...l.mZ...\P@g.M
0160 - 38 18 1d 08 a9 bf 34 32-b7 aa cc 7a d0 26 80 21   8.42...z.&.!
0170 - ee 25 f0 f4 55 c9 5e 96-29 83 32 b7 d9 b7 f8 7e   .%..U.^.).2~
0180 - 6f 11 7c f8 eb dd 3b 05-b9 2e f2 0b 29 1a f5 23   o.|...;.)..#
0190 - 2f 9d 71 a5 77 9e bf ca-2c 34 b1 6b 10 03 b1 1d   /.q.w...,4.k
01a0 - 16 23 12 3c e5 65 5a ab-a3 1c 1d 05 fb f4 d0 b0   .#.<.eZ.
01b0 - a1 f8 0f ec 71 a5 a9 27-96 e2 04 81 50 f4 5d 51   q..'P.]Q
01c0 - 55 32 2e 10 87 ec 8d 2c-9f 5c ef de f9 96 29 25   U2.,.\)%
01d0 - 24 ab 7f 01 56 bf 67 92-ac 34 0f 95 ea 50 15 aa   $...V.g..4...P..
01e0 - 72 0d 02 03 01 00 01 a3-2d 30 2b 30 09 06 03 55   r...-0+0...U
01f0 - 1d 13 04 02 30 00 30 11-06 09 60 86 48 01 86 f8   0.0...`.H...
0200 - 42 01 01 04 04 03 02 04-f0 30 0b 06 03 55 1d 0f   B0...U..
0210 - 04 04 03 02 05 20 30 0d-06 09 2a 86 48 86 f7 0d   . 0...*.H...
0220 - 01 01 0b 05 00 03 82 01-01 00 68 7a 98 cb c4 dd   ..hz
0230 - 08 b3 c0 d4 06 c8 be 5f-e3 29 de 80 b6 91 ee 11   ..._.)..
0240 - 

Re: speeding up bulk loading

[Matthew Von-Maszewski]

Travis,

I cannot address the crash error you see in the logs.  Someone else will have 
to review that problem.

I want to point you to this wiki article:  
https://github.com/basho/leveldb/wiki/riak-tuning-2

The article details how you can potentially increase Riak's throughput by 
restricting the number of schedulers (CPUs) assigned solely to Erlang.  Heavy 
bulk loading scenarios demonstrated 20 to 30% throughput gain.

Per the wiki article, you would create an (or add to your existing) 
advanced.config with the following based upon your 16 core machines:

[
{vm_args, [{"+S", "14:14"}]}
].
Matthew
> On Aug 31, 2016, at 11:41 AM, Travis Kirstine 
>  wrote:
> 
> Magnus
>  
> Thanks for your reply.  We’re are using the riack C client library for riak 
> (https://github.com/trifork/riack ) which 
> is used within an application called MapCache to store 256x256 px images with 
> a corresponding key within riak.  Currently we have 75 million images to 
> transfer from disk into riak which is being done concurrently.  Periodically 
> this transfer process will crash
>  
> Riak is setup using n=3 on 5 nodes with a leveldb backend.  Each server has 
> 45GB of memory and 16 cores with  standard hard drives.  We made no 
> significant modification to the riak.conf except upping the 
> leveldb.maximum_memory.percent to 70 and tweeking the sysctl.conf as follows
>  
> vm.swappiness = 0
> net.ipv4.tcp_max_syn_backlog = 4
> net.core.somaxconn = 4
> net.core.wmem_default = 8388608
> net.core.rmem_default = 8388608
> net.ipv4.tcp_sack = 1
> net.ipv4.tcp_window_scaling = 1
> net.ipv4.tcp_fin_timeout = 15
> net.ipv4.tcp_keepalive_intvl = 30
> net.ipv4.tcp_tw_reuse = 1
> net.ipv4.tcp_moderate_rcvbuf = 1
> # Increase the open file limit
> # fs.file-max = 65536 # current setting
>  
> I have seen this error in the logs
> 2016-08-30 22:26:07.180 [error] <0.20777.512> CRASH REPORT Process 
> <0.20777.512> with 0 neighbours crashed with reason: no function clause 
> matching webmachine_request:peer_from_peername({error,enotconn}, 
> {webmachine_request,{wm_reqstate,#Port<0.2817336>,[],undefined,undefined,undefined,{wm_reqdata,'GET',...},...}})
>  line 150
>  
> Regards
>  
> From: Magnus Kessler [mailto:mkess...@basho.com ] 
> Sent: August-31-16 4:08 AM
> To: Travis Kirstine  >
> Cc: riak-users@lists.basho.com 
> Subject: Re: speeding up bulk loading
>  
> On 26 August 2016 at 22:20, Travis Kirstine  > wrote:
> Is there any way to speed up bulk loading?  I wondering if I should be 
> tweeking the erlang, aae or other config options?
>  
>  
>  
> Hi Travis,
>  
> Excuse the late reply; your message had been stuck in the moderation queue. 
> Please consider subscribing to this list.
>  
> Without knowing more about how you perform bulk uploads, it's difficult to 
> recommend any changes. Are you using the HTTP REST API or one of the client 
> libraries, which use protocol buffers by default? What concerns do you have 
> about the upload performance? Please let us know a bit more about your setup.
>  
> Kind Regards,
>  
> Magnus
>  
>  
> -- 
> Magnus Kessler
> Client Services Engineer
> Basho Technologies Limited
>  
> Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
> ___
> riak-users mailing list
> riak-users@lists.basho.com 
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
> 
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

RE: speeding up bulk loading

[Travis Kirstine]

Magnus

Thanks for your reply.  We’re are using the riack C client library for riak 
(https://github.com/trifork/riack) which is used within an application called 
MapCache to store 256x256 px images with a corresponding key within riak.  
Currently we have 75 million images to transfer from disk into riak which is 
being done concurrently.  Periodically this transfer process will crash

Riak is setup using n=3 on 5 nodes with a leveldb backend.  Each server has 
45GB of memory and 16 cores with  standard hard drives.  We made no significant 
modification to the riak.conf except upping the leveldb.maximum_memory.percent 
to 70 and tweeking the sysctl.conf as follows

vm.swappiness = 0
net.ipv4.tcp_max_syn_backlog = 4
net.core.somaxconn = 4
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_moderate_rcvbuf = 1
# Increase the open file limit
# fs.file-max = 65536 # current setting

I have seen this error in the logs
2016-08-30 22:26:07.180 [error] <0.20777.512> CRASH REPORT Process 
<0.20777.512> with 0 neighbours crashed with reason: no function clause 
matching webmachine_request:peer_from_peername({error,enotconn}, 
{webmachine_request,{wm_reqstate,#Port<0.2817336>,[],undefined,undefined,undefined,{wm_reqdata,'GET',...},...}})
 line 150

Regards

From: Magnus Kessler [mailto:mkess...@basho.com]
Sent: August-31-16 4:08 AM
To: Travis Kirstine 
Cc: riak-users@lists.basho.com
Subject: Re: speeding up bulk loading

On 26 August 2016 at 22:20, Travis Kirstine 
> 
wrote:
Is there any way to speed up bulk loading?  I wondering if I should be tweeking 
the erlang, aae or other config options?



Hi Travis,

Excuse the late reply; your message had been stuck in the moderation queue. 
Please consider subscribing to this list.

Without knowing more about how you perform bulk uploads, it's difficult to 
recommend any changes. Are you using the HTTP REST API or one of the client 
libraries, which use protocol buffers by default? What concerns do you have 
about the upload performance? Please let us know a bit more about your setup.

Kind Regards,

Magnus


--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

RE: Using Riak CS with Hadoop

[Valenti, Anthony]

I did try it with the command below, followed by the error.  I am not sure how 
to specify that it should go to our s3 cluster, but I tried it after the @ sign 
and then the bucket.  It goes to Amazon, which seems to be by default.  I know 
it isn't an issue with Riak and that technically it should work, but I was 
looking for someone that might have tried this and gotten it to work.  It seems 
like a configuration in Hadoop more than anything.  So I am asking there also.

hadoop distcp -update /user/test/client/part-m-0 
s3a://access_key:secret_...@s3.test.com/test-bucket

16/08/31 09:37:38 INFO s3a.S3AFileSystem: Caught an AmazonServiceException, 
which means your request made it to Amazon S3, but was rejected with an error 
response for some reason.
16/08/31 09:37:38 INFO s3a.S3AFileSystem: Error Message: Status Code: 403, AWS 
Service: Amazon S3, AWS Request ID: 94AE01A87CE82C24, AWS Error Code: null, AWS 
Error Message: Forbidden
16/08/31 09:37:38 INFO s3a.S3AFileSystem: HTTP Status Code: 403
16/08/31 09:37:38 INFO s3a.S3AFileSystem: AWS Error Code: null
16/08/31 09:37:38 INFO s3a.S3AFileSystem: Error Type: Client
16/08/31 09:37:38 INFO s3a.S3AFileSystem: Request ID: 94AE01A87CE82C24
16/08/31 09:37:38 INFO s3a.S3AFileSystem: Class Name: 
com.amazonaws.services.s3.model.AmazonS3Exception
16/08/31 09:37:38 ERROR tools.DistCp: Invalid arguments:
com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 403, AWS 
Service: Amazon S3, AWS Request ID: 94AE01A87CE82C24, AWS Error Code: null, AWS 
Error Message: Forbidden, S3 Extended Request ID: 
qYOsZBJ5O3cFaKpbdKEpbou0Zx5hSgsHJHum/kPvjZu6CAvJ2jqVoUVUUXpquEtw
at 
com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
at 
com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
at 
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
at 
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
at 
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:976)
at 
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:956)
at 
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:892)
at 
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at org.apache.hadoop.tools.DistCp.setTargetPathExists(DistCp.java:217)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:116)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:430)
Invalid arguments: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: 
94AE01A87CE82C24, AWS Error Code: null, AWS Error Message: Forbidden


-Original Message-
From: Luke Bakken [mailto:lbak...@basho.com] 
Sent: Wednesday, August 31, 2016 1:18 PM
To: Valenti, Anthony
Cc: riak-users@lists.basho.com
Subject: Re: Using Riak CS with Hadoop

Riak CS provides an S3 capable API, so theoretically it could work.
Have you tried? If so and you're having issues, follow up here.

--
Luke Bakken
Engineer
lbak...@basho.com

On Wed, Aug 31, 2016 at 7:38 AM, Valenti, Anthony  
wrote:
> Has anyone setup Hadoop to be able use Raik CS as an S3 
> source/destination instead of or in addition to Amazon S3?  Hadoop 
> assumes that it should go to Amazon S3 by default.  Specifically, I am 
> trying to use Hadoop distcp to copy files to Riak CS.



Inmar Confidentiality Note:  This e-mail and any attachments are confidential 
and intended to be viewed and used solely by the intended recipient.  If you 
are not the intended recipient, be aware that any disclosure, dissemination, 
distribution, copying or use of this e-mail or any attachment is prohibited.  
If you received this e-mail in error, please notify us immediately by returning 
it to the sender and delete this copy and all attachments from your system and 
destroy any printed copies.  Thank you for your cooperation.

Notice of Protected Rights:  The removal of any copyright, trademark, or 
proprietary legend contained in this e-mail or any attachment is prohibited 
without the express, written permission of Inmar, Inc.  Furthermore, the 
intended recipient must maintain all copyright notices, trademarks, and 
proprietary legends within this e-mail and any attachments in their original 
form and location if the e-mail or any attachments are reproduced, printed or 
distributed.



___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: [ANN] Riak TS v1.4 is released

[Chris.Johnson]

Hey Eric,

Very cool. Thank you!

From: riak-users  on behalf of Eric Johnson 

Date: Monday, August 29, 2016 at 8:34 AM
To: "riak-users@lists.basho.com" 
Subject: Re: [ANN] Riak TS v1.4 is released

Hey Chris, thanks for the heads up.

We’re looking into issues #2 and #4, but we went ahead and fixed #1 and #3 
here. Let us know if 
anything else comes up!

On Fri, Aug 26, 2016 at 2:03 PM, 
> wrote:
We ran this rolling upgrade yesterday and it seemed to mostly work!

A couple problems we encountered:

1.   The command under item 6 on this page 
http://docs.basho.com/riak/ts/1.4.0/setup/upgrading/ did not work for us. We 
had to replace “riak_ts” with “riak_kv”.

2.   There were some transient errors from our clients about the server not 
supporting CRDTs (specifically sets):

a.   Riak::ProtobuffsErrorResponse: Expected success from Riak but received 
0. `set` is not a supported type

3.   The upgrade instructions weren’t clear about the fact that you need to 
update your riak.conf if you have any non-default settings before doing the 
“riak start” step.

4.   There was a unit change for riak_kv.query.timeseries.timeout 
(previously timeseries_query_timeout_ms) in riak.conf that wasn’t clear to us 
from the upgrade instructions. We were under the impression that only the names 
had changed.

chris

From: riak-users 
> 
on behalf of Pavel Hardak >
Date: Thursday, August 25, 2016 at 7:27 PM
To: "riak-users@lists.basho.com" 
>
Cc: Product Team >
Subject: [ANN] Riak TS v1.4 is released


Dear Riak users,


Earlier today we have released Riak TS v1.4. This is well-rounded release, 
combining several new features, number of incremental improvements and bug 
fixes. Please upgrade your Riak TS clusters or if you still did not, take it 
for the spin. Any feedback, good or bad, is welcome. We do listen.



Release notes: http://docs.basho.com/riak/ts/1.4.0/releasenotes/


The new and improved features are:

•  (NEW) GROUP BY statement can be used in SELECT queries. This allows to 
condense the result set using one of Riak TS aggregation functions. [1]

•  (NEW) Users can now use textual date/time representations (per ISO 8601) in 
Riak TS SQL statements, without the need to translate to milliseconds before 
issuing the commands. [2]

•  (NEW) Global object expiration (a.k.a. TTL - time to leave) at the backend 
(LevelDB) level allows efficiently remove aged data from Riak TS. [3]

•  (NEW) Added SHOW TABLES command to list Riak TS tables from Riak Shell. This 
is in addition to riak-admin ‘bucket-type list’, which also works. [4]

•  (NEW) Added official support for Debian 8 Jessie, dropped Debian 6. [6]

•  (IMPROVED) DESCRIBE  command now returns quantum information, in 
addition to the columns names, types and primary key. [5]

•  (IMPROVED) Fine grained security using Riak TS specific permissions. [7]

•  (IMPROVED) Rolling upgrade and downgrade to and from Riak TS 1.3.1. [8]

•  … many documentation and performance improvements.


[1] http://docs.basho.com/riak/ts/1.4.0/using/querying/select/group-by/

[2] http://docs.basho.com/riak/ts/1.4.0/using/timerepresentations/

[3] http://docs.basho.com/riak/ts/1.4.0/using/global-object-expiration/

[4] http://docs.basho.com/riak/ts/1.4.0/using/querying/show-tables/

[5] http://docs.basho.com/riak/ts/1.4.0/downloads/#debian

[6] http://docs.basho.com/riak/ts/1.4.0/using/querying/describe/

[7] http://docs.basho.com/riak/ts/1.4.0/using/security/user-management

[8] http://docs.basho.com/riak/ts/1.4.0/setup/upgrading/


More info:

•  Riak® TS

•  Documentation

•  Basho Blog

•  Downloads

•  Source Code
Pavel Hardak,
Director of Product Management, Basho Technologies

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Using Riak CS with Hadoop

[Luke Bakken]

Riak CS provides an S3 capable API, so theoretically it could work.
Have you tried? If so and you're having issues, follow up here.

--
Luke Bakken
Engineer
lbak...@basho.com

On Wed, Aug 31, 2016 at 7:38 AM, Valenti, Anthony
 wrote:
> Has anyone setup Hadoop to be able use Raik CS as an S3 source/destination
> instead of or in addition to Amazon S3?  Hadoop assumes that it should go to
> Amazon S3 by default.  Specifically, I am trying to use Hadoop distcp to
> copy files to Riak CS.

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Using Riak CS with Hadoop

[Valenti, Anthony]

Has anyone setup Hadoop to be able use Raik CS as an S3 source/destination 
instead of or in addition to Amazon S3?  Hadoop assumes that it should go to 
Amazon S3 by default.  Specifically, I am trying to use Hadoop distcp to copy 
files to Riak CS.

Thanks,
Anthony



Inmar Confidentiality Note:  This e-mail and any attachments are confidential 
and intended to be viewed and used solely by the intended recipient.  If you 
are not the intended recipient, be aware that any disclosure, dissemination, 
distribution, copying or use of this e-mail or any attachment is prohibited.  
If you received this e-mail in error, please notify us immediately by returning 
it to the sender and delete this copy and all attachments from your system and 
destroy any printed copies.  Thank you for your cooperation.

Notice of Protected Rights:  The removal of any copyright, trademark, or 
proprietary legend contained in this e-mail or any attachment is prohibited 
without the express, written permission of Inmar, Inc.  Furthermore, the 
intended recipient must maintain all copyright notices, trademarks, and 
proprietary legends within this e-mail and any attachments in their original 
form and location if the e-mail or any attachments are reproduced, printed or 
distributed.



___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: speeding up bulk loading

[Magnus Kessler]

On 26 August 2016 at 22:20, Travis Kirstine <
tkirst...@firstbasesolutions.com> wrote:

> Is there any way to speed up bulk loading?  I wondering if I should be
> tweeking the erlang, aae or other config options?
>
>
>
>
>
Hi Travis,

Excuse the late reply; your message had been stuck in the moderation queue.
Please consider subscribing to this list.

Without knowing more about how you perform bulk uploads, it's difficult to
recommend any changes. Are you using the HTTP REST API or one of the client
libraries, which use protocol buffers by default? What concerns do you have
about the upload performance? Please let us know a bit more about your
setup.

Kind Regards,

Magnus


-- 
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

speeding up bulk loading

[Travis Kirstine]

Is there any way to speed up bulk loading?  I wondering if I should be tweeking 
the erlang, aae or other config options?


___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com