Re: [akka-user] akka-streams constructing Source from Iterator
Do you mind clarifying what you meant when you mentioned that signature for Source has been changed to take function that creates Iterator in order to share Source between various parts of code? Can this create iterator function be invoked more than once in the same runnable flow execution? If so, I guess I'm wondering what are the circumstances so that I could avoid them, if possible. Andre -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Uniqueness of actors in akka Cluster
You can try Glokka, a library to register and lookup actors by names in an Akka cluster: https://github.com/xitrum-framework/glokka On Monday, March 16, 2015 at 6:06:58 PM UTC+9, Krishna Kadam wrote: Hi all, Right now I am using akka named local actors to send perticutar type of message to the same actor every time, but I am doubtful about its uniqueness in clustering of actors. In simple words If I create and use clustered actors by name and try to send the perticular type of message to same actor every time, will it work in akka cluster? or do you have any other method do this? please help. Thanks and regards Krishna Kadam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Problem with akka tcp streams
Hi!
I want to stream file lines through tcp to server.
Here is the code:
class StreamingTcpActor(remoteAddress:InetSocketAddress, system: ActorSystem){
implicit val actorSystem = system
implicit val materializer = ActorFlowMaterializer()
val connection = StreamTcp().outgoingConnection(remoteAddress)
val file = scala.io.Source.fromFile(new File(./logfile.txt))
val lines = file.getLines()
Source(() = file.getLines()).map(line = {
val message = MessageWithId(UUID.randomUUID().toString, line)
RunTcpStreamEx.messageSerializer.toByteString(message)
}).via(connection.flow).to(Sink.ignore).run()
}
Problem: Some lines were not send to the network.
If I rewrite client code with out akka-streaming everything would be ok.
I am new to akka streams. Could you provide me some doc links to figure out
what`s wrong here.
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] Akka streams, twitter example
http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M4/scala/stream-quickstart.html#stream-quickstart-scala Is there any source code (fully working example, not just snippets) for akka streaming and twitter? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Access to current message in actor
I don't have the time to actually try this now, but one way to achieve this comes to mind: The ActorCell underneath should have access to the message currently in work. AFAIK, you can cast the context to ActorCell and just read the current message out of it. For good reason, however, the ActorCell is an akka-private. - So, you'd have to put the described functionality in the akka packge. I am not sure about the full implications when working with ActorCell, though. ... not a very neat solution, I know. hth -Tom On Wednesday, March 18, 2015 at 11:55:08 PM UTC+1, Eric Pederson wrote: Actually, though, methods in an actor really shouldn't be called asynchronously. That would circumvent the actor model. That's why things like pipeTo exist, to minimize the chance that a future could interact with the guts of an actor. An actor's methods should really only be called from a thread started from receive, with few exceptions. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] akka-streams how to chunk ordered stream based on common property efficiently
Given a stream of elements E ordered by some property, how can I group these elements into List[E] as soon as all all elements with the same property are emitted. I want to continue processing each stream element, which now becomes List[E], as soon as possible. groupBy does not work in this case because it demultiplexes stream elements E based on common property into separate output streams, one for each common property, and it doesn't complete until elements of original stream are emitted. For example, given Source(List(1,1,2,2,2,3,3,3,4)) I want to continue processing List(1,1) as a single stream element as soon as 2 is encountered in the stream. Thank you, Andre Piwoni -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Akka streams, twitter example
The snippets in the documentation are from this file: https://github.com/akka/akka/blob/231a2a0d12c5a96b8b6974ff00839d65f0f7fa6a/akka-docs-dev/rst/scala/code/docs/stream/TwitterStreamQuickstartDocSpec.scala As you can see it is not using real twitter API. There is a blogpost Abhinav Ajgaonkar where he uses real twitter API with akka-streams: http://blog.abhinav.ca/blog/2015/02/19/scaling-with-akka-streams/ On Thu, Mar 19, 2015 at 2:21 AM, Borut borut.daga...@gmail.com wrote: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M4/scala/stream-quickstart.html#stream-quickstart-scala Is there any source code (fully working example, not just snippets) for akka streaming and twitter? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Help with design of Akka cluster and remote workers
Hey Martynas, Thanks for the reply. I'm still getting over the learning curve of Akka and reading the documentation. This looks like what I need, thank you. I have a follow up question. In some of our processes, we require an ack from our receiving actor they have processed the message before we respond to our users. We don't have a fully async system yet, so we have synchronous code invoking asynchronous code. Are there any examples on distributed futures? This way, our older synchronous code can perform a tell to the group, then await it's response before returning to the user. Thanks, Todd On Monday, March 16, 2015 at 11:09:09 AM UTC-6, Martynas Mickevičius wrote: Hello Todd, it seems that routers and particularly cluster aware routers http://doc.akka.io/docs/akka/2.3.9/scala/cluster-usage.html#Cluster_Aware_Routers fit the bill here. Did you have a chance to look at them? On Sun, Mar 15, 2015 at 11:37 AM, Todd Nine tn...@apigee.com javascript: wrote: Hey guys, We're currently using RxJava, and we would like to attempt to integrate Akka as a our distributed work system in the Usergrid project. http://usergrid.incubator.apache.org/ I've gone through a lot of the typesafe activator examples, and they're very helpful. However, there's 1 use case I could use some guidance on. I need the ability for a distributed work system, similar to this example. http://typesafe.com/activator/template/akka-distributed-workers Our flow is the following. 1. User makes an HTTP request, PUT or POST 2. We store the entity they've uploaded into Cassandra 3. We signal a asynchronous worker to start indexing this entity. This will need to be a durable mailbox so if the processing node dies before completion, it will be replayed on another node. We'll probably use the Cassandra persistence here for recovery and durability. 4. The worker begins performing the indexing of the entity to ElasticSearch asynchronously. The example from the typesafe activator uses a master to perform routing to it's workers. I have a few questions. 1. Is it possible to perform the same workflow, but with round robin message distribution and no routing through the master? I'd prefer the message is delivered directly to the worker, if possible. 2. For the actor system, how would I insatiate N workers per node? Must I always have a supervisor actor that then starts workers? I assume so given the actor model, but wanted to ask to be sure. 3. Using the clustering and Gossip, is there some discovery mechanism I can use so that all workers across all nodes appear as one large pool of workers, and the producer does not need any knowledge of the other node's IPs as remote workers? Sorry if these questions are obvious. I'm learning, and most of the examples seem to reference remote actors by IP. If possible, I would prefer to abstract this routing away from our system, and instead have workers appear as one large pool, regardless of location. I'm thinking we'll be starring with Akka 2.3.9 and Java 8. Thanks in advance, Todd -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com javascript:. To post to this group, send email to akka...@googlegroups.com javascript:. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Looking for a good Akka spare time learning project
Am Mittwoch, 18. März 2015 21:29:56 UTC+1 schrieb Ernesto Menéndez: A discrete event simulation engine ( http://en.wikipedia.org/wiki/Discrete_event_simulation). I think this is a good idea and I will take this direction. The game I was thinking of was kind of a discrete event simulation. But I can get a book about it and get input for some serious simulation. Thanks to Eric and Richard for their offers. Think I first have to start with something small to begin with ... -- Haddock -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Transferring data in between cluster nodes
Hi Ömer, it is not recommended to send large messages via the remoting channel. Mainly because the same channel is used for the heartbeat and other system messages. If you are able to use milestone versions of akka-stream and akka-http I would suggest creating an HTTP endpoint where big data structures can be streamed as JSON. Then one would use remoting to exchange the address of the reasourse and akka-http to consume it. On Thu, Mar 19, 2015 at 8:41 AM, Ömer Faruk Gül omergul...@gmail.com wrote: Hi, What would be the best practice to transfer data (an immutable Map which could be of size from 1MB to 100 MB) in between two nodes in a an Akka Cluster? I initially thought of dividing the map into chunks and sending them as a message to the new actor living in the other node, is that a good practice, or should I try something else? My immutable variable is this: var store = Map.empty[String, ByteString] -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] akka-streams constructing Source from Iterator
Hi Andre, the signature has been changed to enable re-materialization of that source. This allows to freely share the source between various parts of the code. I would suggest not only wrapping the ResultSet in a source but also query execution. In that case materializing such a source would run a DB query and fetch results - making it reusable. BTW, I think pre-release versions of Slick 3 come with the reactive-stream interface implementation, which means you can use Slick with akka-streams directly. Did you considered using Slick 3? On Thu, Mar 19, 2015 at 9:31 AM, apiw...@gmail.com wrote: I have been looking at akka-streams 0.7 and there used to be IteratorSource which could be used by passing Iterator. akka-streams 1.0-M4 no longer has IteratorSource but it has a Source object that takes function which produces an Iterator. Is main reason reason for this so that Iterator could be created more than once when multiple Subscribers subscribe? Is there a way to create Source from Iterator that I do not want to create more than once? Basically, given Iterator that wraps ResultSet I don't want akka-streams to create it more than once. I'd rather stay away from implementing Publisher as Slick does in order to ensure once Subscriber. Thanks you, Andre Piwoni -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Transferring data in between cluster nodes
Thanks, I actually make use of spray-can for external data exchange. So I will use it internally too. I have one more question on that but it may be out of scope. I receive POST calls that contains the serialised object which I get as ByteString (extracted from spray.http.HttpEntity) and I simply store it with a key (which I read it from the header). I basically get the ByteString as follows: val bytes: ByteString = request.entity.data.toByteString Now, lets say a node contains an immutable map with million unique keys mapped to ByteStrings. When transferring data to another node, sending a million POST request to the other node could be a problem in terms of performance. Could there be a better way to manage this? On Thursday, March 19, 2015 at 7:48:57 PM UTC+2, Martynas Mickevičius wrote: Hi Ömer, it is not recommended to send large messages via the remoting channel. Mainly because the same channel is used for the heartbeat and other system messages. If you are able to use milestone versions of akka-stream and akka-http I would suggest creating an HTTP endpoint where big data structures can be streamed as JSON. Then one would use remoting to exchange the address of the reasourse and akka-http to consume it. On Thu, Mar 19, 2015 at 8:41 AM, Ömer Faruk Gül omerg...@gmail.com javascript: wrote: Hi, What would be the best practice to transfer data (an immutable Map which could be of size from 1MB to 100 MB) in between two nodes in a an Akka Cluster? I initially thought of dividing the map into chunks and sending them as a message to the new actor living in the other node, is that a good practice, or should I try something else? My immutable variable is this: var store = Map.empty[String, ByteString] -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com javascript:. To post to this group, send email to akka...@googlegroups.com javascript:. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Help with design of Akka cluster and remote workers
Hi Todd, I think you meant using ask, right? You can use ask pattern with cluster aware routers. After asking you will get a future which is going to be completed when some routee in the cluster sends an answer. On Thu, Mar 19, 2015 at 10:04 AM, Todd Nine tn...@apigee.com wrote: Hey Martynas, Thanks for the reply. I'm still getting over the learning curve of Akka and reading the documentation. This looks like what I need, thank you. I have a follow up question. In some of our processes, we require an ack from our receiving actor they have processed the message before we respond to our users. We don't have a fully async system yet, so we have synchronous code invoking asynchronous code. Are there any examples on distributed futures? This way, our older synchronous code can perform a tell to the group, then await it's response before returning to the user. Thanks, Todd On Monday, March 16, 2015 at 11:09:09 AM UTC-6, Martynas Mickevičius wrote: Hello Todd, it seems that routers and particularly cluster aware routers http://doc.akka.io/docs/akka/2.3.9/scala/cluster-usage.html#Cluster_Aware_Routers fit the bill here. Did you have a chance to look at them? On Sun, Mar 15, 2015 at 11:37 AM, Todd Nine tn...@apigee.com wrote: Hey guys, We're currently using RxJava, and we would like to attempt to integrate Akka as a our distributed work system in the Usergrid project. http://usergrid.incubator.apache.org/ I've gone through a lot of the typesafe activator examples, and they're very helpful. However, there's 1 use case I could use some guidance on. I need the ability for a distributed work system, similar to this example. http://typesafe.com/activator/template/akka-distributed-workers Our flow is the following. 1. User makes an HTTP request, PUT or POST 2. We store the entity they've uploaded into Cassandra 3. We signal a asynchronous worker to start indexing this entity. This will need to be a durable mailbox so if the processing node dies before completion, it will be replayed on another node. We'll probably use the Cassandra persistence here for recovery and durability. 4. The worker begins performing the indexing of the entity to ElasticSearch asynchronously. The example from the typesafe activator uses a master to perform routing to it's workers. I have a few questions. 1. Is it possible to perform the same workflow, but with round robin message distribution and no routing through the master? I'd prefer the message is delivered directly to the worker, if possible. 2. For the actor system, how would I insatiate N workers per node? Must I always have a supervisor actor that then starts workers? I assume so given the actor model, but wanted to ask to be sure. 3. Using the clustering and Gossip, is there some discovery mechanism I can use so that all workers across all nodes appear as one large pool of workers, and the producer does not need any knowledge of the other node's IPs as remote workers? Sorry if these questions are obvious. I'm learning, and most of the examples seem to reference remote actors by IP. If possible, I would prefer to abstract this routing away from our system, and instead have workers appear as one large pool, regardless of location. I'm thinking we'll be starring with Akka 2.3.9 and Java 8. Thanks in advance, Todd -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/ current/additional/faq.html Search the archives: https://groups.google.com/ group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com. To post to this group, send email to akka...@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ:
Re: [akka-user] Transferring data in between cluster nodes
That is a one big map. :) Maybe it would be more efficient to use TCP instead of HTTP for such large data structures. Whichever transport you choose you still need to serialize your map to send it through a wire. You can serialize to json (spray-json or play-json support macro based serializer/deserializer generation) if you choose HTTP or you could try scala-pickling, which also supports JSON as well as binary format or some other serialization framework. On Thu, Mar 19, 2015 at 11:25 AM, Ömer Faruk Gül omergul...@gmail.com wrote: Thanks, I actually make use of spray-can for external data exchange. So I will use it internally too. I have one more question on that but it may be out of scope. I receive POST calls that contains the serialised object which I get as ByteString (extracted from spray.http.HttpEntity) and I simply store it with a key (which I read it from the header). I basically get the ByteString as follows: val bytes: ByteString = request.entity.data.toByteString Now, lets say a node contains an immutable map with million unique keys mapped to ByteStrings. When transferring data to another node, sending a million POST request to the other node could be a problem in terms of performance. Could there be a better way to manage this? On Thursday, March 19, 2015 at 7:48:57 PM UTC+2, Martynas Mickevičius wrote: Hi Ömer, it is not recommended to send large messages via the remoting channel. Mainly because the same channel is used for the heartbeat and other system messages. If you are able to use milestone versions of akka-stream and akka-http I would suggest creating an HTTP endpoint where big data structures can be streamed as JSON. Then one would use remoting to exchange the address of the reasourse and akka-http to consume it. On Thu, Mar 19, 2015 at 8:41 AM, Ömer Faruk Gül omerg...@gmail.com wrote: Hi, What would be the best practice to transfer data (an immutable Map which could be of size from 1MB to 100 MB) in between two nodes in a an Akka Cluster? I initially thought of dividing the map into chunks and sending them as a message to the new actor living in the other node, is that a good practice, or should I try something else? My immutable variable is this: var store = Map.empty[String, ByteString] -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/ current/additional/faq.html Search the archives: https://groups.google.com/ group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com. To post to this group, send email to akka...@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Martynas Mickevičius Typesafe http://typesafe.com/ – Reactive http://www.reactivemanifesto.org/ Apps on the JVM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Problem with akka tcp streams
Hi,
are you sure you do not stop this actor or the whole ActorSystem while the
transfer is still going?
On Thu, Mar 19, 2015 at 1:32 AM, zergood zergoodso...@gmail.com wrote:
Hi!
I want to stream file lines through tcp to server.
Here is the code:
class StreamingTcpActor(remoteAddress:InetSocketAddress, system: ActorSystem){
implicit val actorSystem = system
implicit val materializer = ActorFlowMaterializer()
val connection = StreamTcp().outgoingConnection(remoteAddress)
val file = scala.io.Source.fromFile(new File(./logfile.txt))
val lines = file.getLines()
Source(() = file.getLines()).map(line = {
val message = MessageWithId(UUID.randomUUID().toString, line)
RunTcpStreamEx.messageSerializer.toByteString(message)
}).via(connection.flow).to(Sink.ignore).run()
}
Problem: Some lines were not send to the network.
If I rewrite client code with out akka-streaming everything would be ok.
I am new to akka streams. Could you provide me some doc links to figure
out what`s wrong here.
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
Akka User List group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
--
Martynas Mickevičius
Typesafe http://typesafe.com/ – Reactive
http://www.reactivemanifesto.org/ Apps on the JVM
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] akka-streams constructing Source from Iterator
Martynas, I have an existing function that creates Iterator that wraps ResultSet and I can pass this function to Source(f: () = Iterator) which would allow re-materialization. It makes sense though to execute query in hasNext as an optimization in case Iterator is never consumed. Given that I can pass function that creates Iterator wrapper for ResultSet, can I prevent re-materialization of such Source within the same execution of runnable flow. DatabasePublisher from Slick allows only one subscriber so this ensures that there are no concurrency issues. Unfortunately, Slick is not an easy option given existing investment in our database layer. Here's an example of what I would like to accomplish() using akka-streams: Source(() = myDataService.getIterator)) // Iterator.next() returns mapped entity After this point in the flow, is there anything that I need to be aware of that may inadvertently trigger Thank you for your response, Andre -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Transferring data in between cluster nodes
Hi, What would be the best practice to transfer data (an immutable Map which could be of size from 1MB to 100 MB) in between two nodes in a an Akka Cluster? I initially thought of dividing the map into chunks and sending them as a message to the new actor living in the other node, is that a good practice, or should I try something else? My immutable variable is this: var store = Map.empty[String, ByteString] -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] akka-streams constructing Source from Iterator
I have been looking at akka-streams 0.7 and there used to be IteratorSource which could be used by passing Iterator. akka-streams 1.0-M4 no longer has IteratorSource but it has a Source object that takes function which produces an Iterator. Is main reason reason for this so that Iterator could be created more than once when multiple Subscribers subscribe? Is there a way to create Source from Iterator that I do not want to create more than once? Basically, given Iterator that wraps ResultSet I don't want akka-streams to create it more than once. I'd rather stay away from implementing Publisher as Slick does in order to ensure once Subscriber. Thanks you, Andre Piwoni -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
