[akka-user] Re: [Akka-HTTP] How do I not specify the charset?

[André]

Hi Derek,

to get an exact view of the raw incoming request I'd suggest to consult 
Wireshark. Akka's logRequest directive can show slightly different/adapted 
data.

HTH
André

On Tuesday, September 13, 2016 at 7:30:08 PM UTC+2, Derek Wyatt wrote:
>
> I was able to verify that this is breaking the app in question, but not 
> with Akka.  I merely added the charset to the request that was working, and 
> it broke.
>
> But I think I have a more fundamental problem: I can't see the raw request 
> that came in from the client.  It, of course, is only represented using the 
> Akka HTTP model and that model requires a charset for NonBinary 
> ContentTypes.  So, if I wanted to try and force the outgoing request (to 
> the server for which I am being a reverse-proxy) to match the incoming 
> request in this regard, I wouldn't have enough information to know that I 
> should do that.
>
> Is there any way to get at the raw HTTP request so I can inspect it?  I 
> think I need to see exactly how the client created it in the first place.
>
> On Tuesday, September 13, 2016 at 12:58:15 PM UTC-4, Derek Wyatt wrote:
>>
>> Hi guys,
>>
>> I've written a reverse-proxy with Akka-HTTP and I'm hitting a problem 
>> while trying to interact with an ASP.NET app.  The only interesting 
>> thing I can see is that without the proxy, the Content-Type is lacking a 
>> charset but the proxy adds charset=UTF-8.
>>
>> I can't find a way to replicate that using Akka HTTP.  I think we can all 
>> agree that the addition of charset=UTF-8 is the right thing to do, but in 
>> this case it may be causing us a problem and I'd like to force it to look 
>> the same.  The best I've gotten so far is a definition where it has 
>> 'charset=', which isn't quite right.
>>
>> Can this be done?
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Multithreading on Akka-Cluster

[Guido Medina]

What you have seen for Akka Artery M4 is what is available at the moment,
I did test it when it was M2 but I haven't done anything with M4 yet,
for M4 I imagine the same applies as it was for M4, the release notes is 
the guide.

Have you measured your application like you were suggested?
I imagine you can at least log how many ms takes to query from CouchDB?
I also imagine at Node 1 your actors querying from CouchDB are running on a 
separate dispatcher because any DB operation is considered to be a blocking 
operation?

I believe you need to figure out where is your bottleneck first.

HTH,

Guido.

On Tuesday, September 13, 2016 at 3:28:15 PM UTC+1, silvio poma wrote:
>
> Hi Guido, can you suggest me some guide that explains the configuration 
> changes that need to be done?
> I've seen that with the M4 it was included the configuration for for 
> usage with Docker, that is what i need.
>
> Thanks Silvio
>
> Il giorno mercoledì 7 settembre 2016 17:50:44 UTC+2, Guido Medina ha 
> scritto:
>>
>> Artery is at  in milestone 3 (m3), you are welcome to use as I believe it 
>> is in an extremely good shape but I would pursue both,
>> as it will only require you a day of work (as a newbie) in configuration 
>> and dependencies, no code changes will be required:
>>
>>
>> http://search.maven.org/#artifactdetails%7Ccom.typesafe.akka%7Cakka-remote_2.11%7C2.4-ARTERY-M3%7Cjar
>>
>> As for dispatchers, be careful there, I didn't mean "one dispatcher per 
>> actor" but more like "one dispatcher per one or more types of actors"
>>
>> HTH,
>>
>> Guido.
>>
>> On Wednesday, September 7, 2016 at 4:37:13 PM UTC+1, silvio poma wrote:
>>>
>>> Dear Guido,
>>> Thanks a lot for you answer. 
>>> We have already improved our concurrency model, using multiple instances 
>>> for each actor, that now work in parallel using multiple dispatcher.
>>> Now we have not so much time to release our product to the client. 
>>> What do you suggest to focus our efforts on? 
>>> Which of this two arguments can effectively improve the system 
>>> efficiency?
>>>
>>> - Passing from Remote to Artery?
>>> - Remove Java serializer and use another one?
>>>
>>> Thanks,
>>> Looking forward to hear you.
>>> Silvio
>>>   
>>>
>>> Il giorno mercoledì 7 settembre 2016 11:16:08 UTC+2, Guido Medina ha 
>>> scritto:

 Messages can be slow when being sent remotely from node A to node B,
 current Akka remote will give you a top of 100k msg/sec best scenario 
 and that depending on the message size,
 there is a new Akka remote (Akka artery which is a rewrite of Akka 
 remote) on the way so that shouldn't discourage you.

 Serialization is important, make sure you are not using the default 
 Java serialization, I suggest you to use Kryo for example.

 The fact that you are using actors doesn't really mean you are using 
 concurrency right, books like "Java concurrency in Practice" still apply, 
 you need to know how dispatchers work in Akka
 and how to spread your actors among dispatcher for specialized tasks, 
 also, you need to learn how to create several instances of the same actor 
 so that they act as workers,
 hence activating a proper level of concurrency.

 I think the main problem is that *-it is a common mistake by new Akka 
 users-* is to try to just throw actors at the problem and expect that 
 it will magically solve concurrency issues.
 You need to read the documentation and understand how actors, routers 
 and dispatchers can work together and how to combine them.

 If you are simply sending messages to one actor in a node you still 
 have a single threaded kind processor,

 I have tackled this kind of problem by creating several instances *-say 
 N x CPUs depending on the problem they are solving-*,
 send such list to a remote node and there create a dumb round-robin 
 router and use such router to send messages to such pool of actors,
 and then you will have some decent concurrency in that node, make sure 
 you use more than one dispatcher or else everything will be executed by 
 the 
 same dispatcher,
 which translates to having a single thread pool executor doing 
 everything.

 http://doc.akka.io/docs/akka/2.4.10/java/dispatchers.html
 http://doc.akka.io/docs/akka/2.4.10/java/routing.html

 HTH,

 Guido.

 On Monday, September 5, 2016 at 11:58:47 AM UTC+1, silvio poma wrote:
>
> Dear Konrad,
> Thanks for your answer, our Actors didn't do anything that blocks the 
> system.
> One possible cause for our performance problem is that in our 
> "cluster-metrics-adaptive-group" router have just one routees associated 
> to 
> one actor (We have one actor for microsistem right now). We think to 
> increase the number of actor for each microsistem and consequencialy the 
> routees lists, is this a possible solution?
> If it is righ

Re: [akka-user] Akka Multithreading on Akka-Cluster

[silvio poma]

Hi Guido,
Following the release notes i create a config file like this:


"remote": {
 "adapters": {
 "gremlin": "akka.remote.transport.FailureInjectorProvider",
 "trttl": "akka.remote.transport.ThrottlerProvider"
 },
 "artery": {
 "bind-hostname": "127.0.1.1",
 "enabled": "on",
 "hostname": "127.0.1.1",
 "tcp": {
 "bind-port": 2551,
 "port": 2551
 }
 },
 "backoff-interval": "5 ms",
 "backoff-remote-dispatcher": {
 "executor": "fork-join-executor",
 "fork-join-executor": {
 "parallelism-max": 2,
 "parallelism-min": 2
 },
 "type": "Dispatcher"
 },
 "command-ack-timeout": "30 s",
 "default-remote-dispatcher": {
 "executor": "fork-join-executor",
 "fork-join-executor": {
 "parallelism-max": 2,
 "parallelism-min": 2
 },
 "type": "Dispatcher"
 },
 "enabled-transports": ["akka.remote.netty.tcp"],
 "flush-wait-on-shutdown": "2 s",
 "gremlin": {
 "debug": "off"
 },
 "initial-system-message-delivery-timeout": "3 m",
 "log-buffer-size-exceeding": 5,
 "log-frame-size-exceeding": "off",
 "log-received-messages": "off",
 "log-remote-lifecycle-events": "off",
 "log-sent-messages": "off",
 "netty": {
 "ssl": {
 "applied-adapters": [],
 "backlog": 4096,
 "bind-hostname": "",
 "bind-port": "",
 "client-socket-worker-pool": {
 "pool-size-factor": 1,
 "pool-size-max": 2,
 "pool-size-min": 2
 },
 "connection-timeout": "15 s",
 "enable-ssl": true,
 "hostname": "",
 "maximum-frame-size": "128000b",
 "port": 2552,
 "receive-buffer-size": "256000b",
 "security": {
 "enabled-algorithms": ["TLS_RSA_WITH_AES_128_CBC_SHA"],
 "key-password": "changeme",
 "key-store": "keystore",
 "key-store-password": "changeme",
 "protocol": "TLSv1",
 "random-number-generator": "",
 "trust-store": "truststore",
 "trust-store-password": "changeme"
 },
 "send-buffer-size": "256000b",
 "server-socket-worker-pool": {
 "pool-size-factor": 1,
 "pool-size-max": 2,
 "pool-size-min": 2
 },
 "tcp-keepalive": "on",
 "tcp-nodelay": "on",
 "tcp-reuse-addr": "off-for-windows",
 "transport-class": "akka.remote.transport.netty.NettyTransport",
 "transport-protocol": "tcp",
 "use-dispatcher-for-io": "",
 "write-buffer-high-water-mark": "0b",
 "write-buffer-low-water-mark": "0b"
 },
 "tcp": {
 "applied-adapters": [],
 "backlog": 4096,
 "bind-hostname": "",
 "bind-port": "",
 "client-socket-worker-pool": {
 "pool-size-factor": 1,
 "pool-size-max": 2,
 "pool-size-min": 2
 },
 "connection-timeout": "15 s",
 "enable-ssl": false,
 "hostname": "",
 "maximum-frame-size": "128000b",
 "port": 2552,
 "receive-buffer-size": "256000b",
 "send-buffer-size": "256000b",
 "server-socket-worker-pool": {
 "pool-size-factor": 1,
 "pool-size-max": 2,
 "pool-size-min": 2
 },
 "tcp-keepalive": "on",
 "tcp-nodelay": "on",
 "tcp-reuse-addr": "off-for-windows",
 "transport-class": "akka.remote.transport.netty.NettyTransport",
 "transport-protocol": "tcp",
 "use-dispatcher-for-io": "",
 "write-buffer-high-water-mark": "0b",
 "write-buffer-low-water-mark": "0b"
 },
 "udp": {
 "applied-adapters": [],
 "backlog": 4096,
 "bind-hostname": "",
 "bind-port": "",
 "client-socket-worker-pool": {
 "pool-size-factor": 1,
 "pool-size-max": 2,
 "pool-size-min": 2
 },
 "connection-timeout": "15 s",
 "enable-ssl": false,
 "hostname": "",
 "maximum-frame-size": "128000b",
 "port": 2552,
 "receive-buffer-size": "256000b",
 "send-buffer-size": "256000b",
 "server-socket-worker-pool": {
 "pool-size-factor": 1,
 "pool-size-max": 2,
 "pool-size-min": 2
 },
 "tcp-keepalive": "on",
 "tcp-nodelay": "on",
 "tcp-reuse-addr": "off-for-windows",
 "transport-class": "akka.remote.transport.netty.NettyTransport",
 "transport-protocol": "udp",
 "use-dispatcher-for-io": "",
 "write-buffer-high-water-mark": "0b",
 "write-buffer-low-water-mark": "0b"
 }
 },



and i had this error:

No transport is loaded for protocol: [artery], available protocols: [akka.
tcp]
akka.remote.RemoteTransportException: No transport is loaded for protocol: [
artery], available protocols: [akka.tcp]


Anyway yes i measured the CouchDB query it's go from 30 to 300 ms more or 
less. And i implemented it with future so it should be a blocking 
operation, am i right?

Thanks!

Il giorno mercoledì 14 settembre 2016 10:31:43 UTC+2, Guido Medina ha 
scritto:
>
> What you have seen for Akka Artery M4 is what is available at the moment,
> I did test it when it was M2 but I haven't done anything with M4 yet,
> for M4 I imagine the same applies as it was for M4, the release notes is 
> the guide.
>
> Have you measured your application like you were suggested?
> I imagine you can at least log how many ms takes to query from CouchDB?
> I also imagine at Node 1 your actors querying from CouchDB are running on 
> a separate dispatcher because any DB operation is considered to be a 
> blocking operation?
>
> I believe you need to figure out where is your bottleneck first.
>
> HTH,
>
> Guido.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>

Re: [akka-user] Enjoying Akka HTTP performance

[Guido Medina]

If you want to test Netty indirectly and have some reasonable API in the 
middle try comparing with Vert.x 3.
That way you are not just comparing against Netty but against a framework 
that is built on top of Netty,
has a promising HTTP API and is matured (also faster than NodeJS of course):

http://vertx.io/docs/vertx-core/java/#_creating_an_http_server

Regards,

Guido.

On Monday, September 12, 2016 at 2:09:36 PM UTC+1, Konrad Malawski wrote:
>
> That's very cool - thanks for posting these Christian!
> We didn't actually compare with Play, didn't have time somehow; We were 
> focused on beating Spray :-)
>
> Very cool to see we're on par with Netty Play.
>
> -- 
> Konrad `ktoso` Malawski
> Akka  @ Lightbend 
>
> On 12 September 2016 at 15:08:11, Christian Schmitt (
> c.sc...@briefdomain.de ) wrote:
>
> Reflog: 
>
> schmitch@deployster:~/projects/schmitch/wrk2$ git reflog HEAD
>
> c4250ac HEAD@{0}: clone: from https://github.com/giltene/wrk2.git
>
> Am Montag, 12. September 2016 15:07:08 UTC+2 schrieb Christian Schmitt: 
>>
>> it is actually wrk2: 
>>
>> schmitch@deployster:~/projects/schmitch/wrk2$ ./wrk --version
>>
>> wrk 4.0.0 [kqueue] Copyright (C) 2012 Will Glozer
>>
>>
>> I compiled it on the mac against the homebrew openssl library.
>>
>> Actually I also thing that at something like 60k-70k packages my client 
>> network gear and the switch starts to fall behind (thats why the latency is 
>> so high).
>>
>> Am Montag, 12. September 2016 15:01:40 UTC+2 schrieb √: 
>>>
>>> https://github.com/giltene/wrk2
>>>
>>> On Mon, Sep 12, 2016 at 2:59 PM, Christian Schmitt <
>>> c.sc...@briefdomain.de> wrote:
>>>
 extracted from my gist:

 akka-http:
 schmitch@deployster:~/projects/schmitch/wrk2$ ./wrk -t2 -c100 -d300s 
 -R120k http://192.168.179.157:3000
 Running 5m test @ http://192.168.179.157:3000
   2 threads and 100 connections
   Thread calibration: mean lat.: 787.360ms, rate sampling interval: 
 2975ms
   Thread calibration: mean lat.: 585.613ms, rate sampling interval: 
 2473ms
   Thread Stats   Avg  Stdev Max   +/- Stdev
 Latency30.11s22.42s1.58m62.48%
 Req/Sec44.77k 4.77k   54.28k58.88%
   26888534 requests in 5.00m, 4.50GB read
 Requests/sec:  89628.49
 Transfer/sec: 15.37MB

 play with netty and native enabled (netty without native is exactly the 
 same as akka-http:
 schmitch@deployster:~/projects/schmitch/wrk2$ ./wrk -t2 -c100 -d300s 
 -R120k http://192.168.179.157:9000
 Running 5m test @ http://192.168.179.157:9000
   2 threads and 100 connections
   Thread calibration: mean lat.: 625.068ms, rate sampling interval: 
 2504ms
   Thread calibration: mean lat.: 696.276ms, rate sampling interval: 
 2562ms
   Thread Stats   Avg  Stdev Max   +/- Stdev
 Latency28.14s18.49s1.32m61.39%
 Req/Sec46.78k 3.23k   51.52k52.63%
   28079997 requests in 5.00m, 4.02GB read
 Requests/sec:  93600.05
 Transfer/sec: 13.74MB

























 Am Montag, 12. September 2016 14:52:48 UTC+2 schrieb √: 
>
> What does wrk2 say?
>
> On Mon, Sep 12, 2016 at 2:37 PM, Christian Schmitt <
> c.sc...@briefdomain.de> wrote:
>
>> I just compared Playframework on Netty vs Akka-http guess thats fair 
>> since play is quite high level. 
>>
>> Performance for 2k, 20k, 120k Req/s (2k was used to warmup the VM): 
>> https://gist.github.com/schmitch/2ca3359bc34560c6063d0b00eb0a7aac
>> Projects: https://github.com/schmitch/performance (akka-http is just 
>> his project + @volatile on the var)
>>
>> Am Montag, 12. September 2016 13:04:29 UTC+2 schrieb Konrad Malawski: 
>>>
>>>
>>>
>>> -- 
>>> Konrad `ktoso` Malawski
>>> Akka  @ Lightbend 
>>>
>>> On 12 September 2016 at 12:56:46, Christian Schmitt (
>>> c.sc...@briefdomain.de) wrote:
>>>
>>> actually wouldn't it be more reasonable to try it against netty?
>>>
>>> Yes and no. Then one should compare raw IO APIs, and none of the 
>>> high-level features Akka HTTP provides (routing, trivial back-pressured 
>>> entity streaming, fully typesafe http model) etc. 
>>>
>>> It's a fun experiment to see how much faster Netty is, but I don't 
>>> think it's the goal here – if you really want to write each and every 
>>> microservice with raw Netty APIs–enjoy, but I don't think that's the 
>>> nicest 
>>> API to just bang out a service in 4 minutes :)
>>>
>>> (Note, much love for Netty here, but I don't think comparing 1:1 
>>> with Akka HTTP here is the right way to look at it (yes, of course 
>>> we'll be 
>>> slower ;-)).
>

Re: [akka-user] Akka Multithreading on Akka-Cluster

[Guido Medina]

Hi Silvio,

The following are the only things I can recommend you:

   - You have a short time to deliver? Why bother now with Artery if that's 
   the least of your problems? Your performance issue has nothing to do with 
   Akka Cluster or Akka Remote.
   - Why use a future? Your future is probably running at the default 
   dispatcher hence you are blocking at the wrong place !!!
   - Why don't you just dedicate a dispatcher for your worker actors and 
   just do your blocking call there with no future, a standard synchronous 
   call, in the worst scenarios messages will just accumulate at your workers 
   which is OK, you can set some expire time to not deal with then if they 
   have expired but please keep it simple, make it work at a decent speed and 
   then make it fast.

Try that and it should improve your performance, before you try more 
complex scenarios please read the Akka documentation or you will be slowing 
things rather than making them faster, start simple then add to it.

HTH,

Guido.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Multithreading on Akka-Cluster

[Patrik Nordwall]

The Artery config is not correct, see release announcement for M4

/Patr
ons 14 sep. 2016 kl. 14:12 skrev Guido Medina :

> Hi Silvio,
>
> The following are the only things I can recommend you:
>
>- You have a short time to deliver? Why bother now with Artery if
>that's the least of your problems? Your performance issue has nothing to do
>with Akka Cluster or Akka Remote.
>- Why use a future? Your future is probably running at the default
>dispatcher hence you are blocking at the wrong place !!!
>- Why don't you just dedicate a dispatcher for your worker actors and
>just do your blocking call there with no future, a standard synchronous
>call, in the worst scenarios messages will just accumulate at your workers
>which is OK, you can set some expire time to not deal with then if they
>have expired but please keep it simple, make it work at a decent speed and
>then make it fast.
>
> Try that and it should improve your performance, before you try more
> complex scenarios please read the Akka documentation or you will be slowing
> things rather than making them faster, start simple then add to it.
>
> HTH,
>
> Guido.
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] combining Akka Actors, Streams, Cassandra Persistence

[Dagny T]

Has anyone tried this successfully yet?

I'm interested in combining the:
* Akka Reactive Streams Async API goodness
* along with the Actor State-Machine goodness
* and the Cassandra Dynamic Partitioning Fault-Tolerant Cluster goodness

I am running into an Architecture understanding issue in that I really like 
the Cassandra Cluster Gossip solution;

but I'm feeling like it would even make sense for my Akka Actors to have a 
similar solution for the App API level;
i.e. instead of having to hardcode IP addresses for Akka Actor Cluster 
nodes in an Application.conf (which is a hassle on adding-removing Nodes);
or having to figure out Startup/Shutdown protocols for an essentially 
Hierarchical Master-Slave architecture which likely has SPOF issues.

Does anyone from Akka Team have any insight into how best to think about 
putting these elements together -- or if I should be pursuing a different 
combination of tech that serves the same purpose?

THANKs in advance!

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Is back pressure triggered upon exceptions

[Kunal Deshpande]

Hi, 

I have been using Akka streams to implement a saved-search refresh system 
as well as a notification processing system at my current company. 

Recently we ran into a fast-publisher & slow subscriber problem where the 
downstream HTTP services were taking a long time to respond resulting in 
Timeout exceptions in our client. Currently we simply drop the event and 
resume the stream using Supervision.Resume but I am unsure whether that 
translates into back pressure.

Few questions on back pressure
1. While using Flows in akka-streams using .via will a downstream flow 
apply back pressure to a flow upstream or is back pressure only signaled to 
a Source?
2. Will exceptions in a Flow trigger back pressure
3. Is there a mathematical way to represent back pressure and is it 
consistent across different reactive streams implementations?

Thanks, and really appreciate your time!

Kunal

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] [Akka-Streams] Using Framing.delimiter where ByteString stream is part of a Tuple

[Paul Brown]

Hi Akka Stream Users,

I'm fairly new to Akka Streams, and only just getting starting 
understanding some of its concepts. 

One problem I'm having is trying to understand how Tuples are handled in a 
stream. In my scenario I have a Source emitting Tuple2 of (ZipEntry(), 
ByteString()). ZipEntry contains basic information (name, date created) 
about the zip file, and the ByteString contains the extracted text of the 
zip file.

I'm comfortable getting a stream performing a delimit of a ByteString with 
the following.
Source.map(_._2).via(Framing.delimiter(ByteString("\n"), Int.MaxValue)))

However, what I would like to be able to do is maintain the ZipEntry 
information alongside the ByteString for each new row created through the 
delimiter. That way I can process the stream further using groupBy etc. 

Example Source --> Sink

Source input:
(ZipEntry(name1.txt, date1), ByteString(58, 49, ..., 54, 58, ..., 48, 48, 
...) )
(ZipEntry(name2.txt, date2), ByteString(49, 58, ..., 58, 54, ..., 48, 48, 
...) )


Sink output:
(ZipEntry(name1.txt, date), ByteString(58, 49, ...) )
(ZipEntry(name1.txt, date), ByteString(54, 58, ...) )
(ZipEntry(name1.txt, date), ByteString(48, 48, ...) )
...
(ZipEntry(name2.txt, date), ByteString(58, 49, ...) )
(ZipEntry(name2.txt, date), ByteString(54, 58, ...) )
(ZipEntry(name2.txt, date), ByteString(48, 48, ...) )
...

Should I be looking at a breaking it down into a RunnableGraph? 
Any help/pointers to documentation on the correct approach would be 
appreciated.

Thanks
Paul

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.