Hi Yi,
Thanks for the reply. I tried to test the upgrading locally with our toy
job (ThreadJob this time, not Yarn job, so the YARN version shouldn't
matter).
And change the jackson version back to 1.8.5.
And now I am getting the serialization error in the JSON serde:
2016-03-28 11:46:23 SamzaContainer [ERROR] Caught exception in process loop.
org.apache.samza.SamzaException: can not serialize the message
at
org.apache.samza.system.SystemProducers.send(SystemProducers.scala:76)
at
org.apache.samza.task.TaskInstanceCollector.send(TaskInstanceCollector.scala:61)
at com.uber.athena.TripCounterTask.window(TripCounterTask.java:149)
at
org.apache.samza.container.TaskInstance$$anonfun$window$1.apply$mcV$sp(TaskInstance.scala:166)
at
org.apache.samza.container.TaskInstanceExceptionHandler.maybeHandle(TaskInstanceExceptionHandler.scala:54)
at
org.apache.samza.container.TaskInstance.window(TaskInstance.scala:165)
at
org.apache.samza.container.RunLoop$$anonfun$window$1$$anonfun$apply$mcVJ$sp$5.apply(RunLoop.scala:146)
at
org.apache.samza.container.RunLoop$$anonfun$window$1$$anonfun$apply$mcVJ$sp$5.apply(RunLoop.scala:143)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
at
org.apache.samza.container.RunLoop$$anonfun$window$1.apply$mcVJ$sp(RunLoop.scala:143)
at
org.apache.samza.util.TimerUtils$class.updateTimerAndGetDuration(TimerUtils.scala:51)
at
org.apache.samza.container.RunLoop.updateTimerAndGetDuration(RunLoop.scala:35)
at org.apache.samza.container.RunLoop.window(RunLoop.scala:137)
at org.apache.samza.container.RunLoop.run(RunLoop.scala:75)
at
org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:553)
at org.apache.samza.job.local.ThreadJob$$anon$1.run(ThreadJob.scala:42)
Caused by: org.apache.samza.SamzaException:
org.codehaus.jackson.map.JsonMappingException: No serializer found for
class java.util.HashMap$Entry and no properties discovered to create
BeanSerializer (to avoid exception, disable
SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) )
at org.apache.samza.serializers.JsonSerde.toBytes(JsonSerde.scala:36)
at
org.apache.samza.serializers.SerdeManager.toBytes(SerdeManager.scala:69)
at
org.apache.samza.system.SystemProducers.send(SystemProducers.scala:74)
... 15 more
Caused by: org.codehaus.jackson.map.JsonMappingException: No
serializer found for class java.util.HashMap$Entry and no properties
discovered to create BeanSerializer (to avoid exception, disable
SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) )
at
org.codehaus.jackson.map.ser.StdSerializerProvider$1.failForEmpty(StdSerializerProvider.java:89)
at
org.codehaus.jackson.map.ser.StdSerializerProvider$1.serialize(StdSerializerProvider.java:62)
at
org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:600)
at
org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:280)
at
org.codehaus.jackson.map.ObjectMapper._configAndWriteValue(ObjectMapper.java:2260)
at
org.codehaus.jackson.map.ObjectMapper.writeValueAsString(ObjectMapper.java:1829)
at org.apache.samza.serializers.JsonSerde.toBytes(JsonSerde.scala:33)
... 17 more
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down consumer multiplexer.
2016-03-28 11:46:23 BrokerProxy [INFO] Shutting down BrokerProxy for
localhost:9092
2016-03-28 11:46:23 BrokerProxy [INFO] closing simple consumer...
2016-03-28 11:46:23 BrokerProxy [INFO] Shutting down due to interrupt.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down task instance
stream tasks.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down task instance stores.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down producer multiplexer.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down locality manager.
2016-03-28 11:46:23 CoordinatorStreamSystemProducer [INFO] Stopping
coordinator stream producer.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down offset manager.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down metrics reporters.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutting down JVM metrics.
2016-03-28 11:46:23 SamzaContainer [INFO] Shutdown complete.
So it cannot serialize HashMap entry, which in 0.9 works fine. I checked
the JsonSerde class in 0.10, and found out it replaced the codehaus
ObjectMapper (used in 0.9) with a SamzaObjectMapper (a wrapper on top if
codehaus ObjectMapper). Not sure if that will affect. Curious does anyone
else encountered this problem while upgrading?
Thanks!
Yuanchi
On Wed, Mar 23, 2016 at 3:50 PM, Yi Pan <nickpa...@gmail.com> wrote:
> Hi, Yuanchi,
>
> At LinkedIn, we have teams using Samza 0.10 with jackson-1.8.5 packaged
> within. We have not heard any issue w/ that. One difference though: the
> minimum supported (i.e. tested YARN version) in Samza 0.10 is 2.6.1 and
> LinkedIn is running 2.7.1.
>
> -Yi
>
> On Wed, Mar 23, 2016 at 3:15 PM, Yuanchi Ning <yuan...@uber.com> wrote:
>
> > But by in 0.10 release it's using 1.8.5 by default, you mean that other
> > people are fine with this version when upgrading?
> >
> > On Wed, Mar 23, 2016 at 3:13 PM, Yuanchi Ning <yuan...@uber.com> wrote:
> >
> >> Hi Yi,
> >>
> >> This is aonther issue. I overwrote it in our rapper library called
> >> athena-core-lib. The mvn dependency tree shows that it's picking up the
> >> version 1.9.13, plus the command line that runs the program shows
> 1.9.13.
> >> We are currently testing on different Yarn versions to see if that's
> >> related since our current version is 2.6.0.
> >>
> >> Thanks!
> >> Yuanchi
> >>
> >> On Wed, Mar 23, 2016 at 2:57 PM, Yi Pan <nickpa...@gmail.com> wrote:
> >>
> >>> Hi, Yuanchi,
> >>>
> >>> Is this related w/ the issue you reported earlier regarding to "problem
> >>> picking up checkpoint after upgrade" in another thread? I assume that
> you
> >>> are using the official Samza 0.10 release? That has jackson version
> 1.8.5
> >>> by default. How do you change it in your own build/package to 1.9.13?
> >>>
> >>> Thanks!
> >>>
> >>> -Yi
> >>>
> >>> On Wed, Mar 23, 2016 at 12:00 PM, Yuanchi Ning <
> ningyuanchi...@gmail.com
> >>> > wrote:
> >>>
> >>>> Hi All,
> >>>>
> >>>> When we upgrade our existing samza job from 0.9 to 0.10, we found this
> >>>> error below:
> >>>>
> >>>>
> >>>>
> /var/lib/hadoop-yarn/data/samza-yarn/usercache/yuanchi/appcache/application_1458330755534_0075/container_1458330755534_0075_02_000001/__package/bin/run-class.sh:
> >>>> line 63: OpenJDK: command not found
> >>>> java version "1.7.0_95"
> >>>> OpenJDK Runtime Environment (IcedTea 2.6.4)
> >>>> (7u95-2.6.4-0ubuntu0.12.04.1)
> >>>> OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)
> >>>> Exception in thread "main" java.lang.NullPointerException
> >>>> at java.io.StringReader.<init>(StringReader.java:50)
> >>>> at
> >>>>
> org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:636)
> >>>> at
> >>>>
> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863)
> >>>> at
> >>>>
> org.apache.samza.job.yarn.SamzaAppMaster$.main(SamzaAppMaster.scala:71)
> >>>> at
> >>>> org.apache.samza.job.yarn.SamzaAppMaster.main(SamzaAppMaster.scala)
> >>>>
> >>>>
> >>>> And that line 71 in SamzaAppMaster is:
> >>>>
> >>>> val coordinatorSystemConfig = new
> >>>>
> >>>>
> MapConfig(SamzaObjectMapper.getObjectMapper.readValue(System.getenv(ShellCommandConfig.ENV_COORDINATOR_SYSTEM_CONFIG),
> >>>> classOf[Config]))
> >>>>
> >>>> And our checkpoint/ new version related properties are as below:
> >>>>
> >>>> ##################### Job config #####################
> >>>>
> >>>> job.factory.class=org.apache.samza.job.yarn.YarnJobFactory
> >>>>
> >>>> job.name=trip-counter
> >>>>
> >>>> job.datacenter=sjc1
> >>>>
> >>>> job.environment=sandbox
> >>>>
> >>>> job.coordinator.system=kafka
> >>>>
> >>>> job.coordinator.replication.factor=3
> >>>>
> >>>>
> >>>> ##################### Task config #####################
> >>>>
> >>>> task.class=com.uber.athena.TripCounterTask
> >>>>
> >>>> task.inputs=kafka.trip_details,kafka.hp-api-client_signups
> >>>>
> >>>> task.outputTripTopic=trip_count_details
> >>>>
> >>>> task.outputClientSignUpsTopic=client_sign_ups_count_details
> >>>>
> >>>>
> >>>>
> task.checkpoint.factory=org.apache.samza.checkpoint.kafka.KafkaCheckpointManagerFactory
> >>>>
> >>>> task.checkpoint.system=kafka
> >>>>
> >>>> task.checkpoint.replication.factor=3
> >>>>
> >>>>
> >>>> Plus our version for codehaus jackson dependencies is 1.9.13 as
> >>>> suggested in other dev email threads.
> >>>>
> >>>>
> >>>> Any ideas what would cause this issue?
> >>>>
> >>>> Thanks!
> >>>>
> >>>> Yuanchi
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Yuanchi Ning
> >>>>
> >>>
> >>>
> >>
> >
>
--
Yuanchi Ning
Master of Information Technology
Very Large Information System
School of Computer Science
Carnegie Mellon University
Mobile: (412)680-9774
Email: ningyuanchi...@gmail.com
yuanc...@cs.cmu.edu
yuanc...@andrew.cmu.edu
