This looks like the output from the client - do you have some TaskManager log files with more log entries?
That would be helpful... On Wed, Jan 3, 2018 at 5:26 PM, Kyle Hamlin <hamlin...@gmail.com> wrote: > Hello Stephan & Nico, > > Here is the full stacktrace, its not much more than what I originally > posted. I remember seeing an XMLInputFactory input error at one point, > but I haven't seen that again. Is there any other information I can provide > that will help resolve? > > [flink-1.4.0] ./bin/flink run ~/streaming.jar > Cluster configuration: Standalone cluster with JobManager at localhost/ > 127.0.0.1:6123 > Using address localhost:6123 to connect to JobManager. > JobManager web interface address http://localhost:8082 > Starting execution of program > Submitting job with JobID: 022e4a310cd56ba4f7befc0286921663. Waiting for > job completion. > Connected to JobManager at > Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-1321297259] > with leader session id 00000000-0000-0000-0000-000000000000. > 01/03/2018 11:21:53 Job execution switched to status RUNNING. > 01/03/2018 11:21:53 Source: Kafka -> Sink: Unnamed(1/1) switched to > SCHEDULED > 01/03/2018 11:21:53 Source: Kafka -> Sink: Unnamed(1/1) switched to > DEPLOYING > 01/03/2018 11:21:53 Source: Kafka -> Sink: Unnamed(1/1) switched to > RUNNING > 01/03/2018 11:21:54 Source: Kafka -> Sink: Unnamed(1/1) switched to FAILED > java.lang.NoClassDefFoundError: Could not initialize class > org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.internal. > S3ErrorResponseHandler > at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services. > s3.AmazonS3Client.<init>(AmazonS3Client.java:363) > at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services. > s3.AmazonS3Client.<init>(AmazonS3Client.java:542) > at org.apache.flink.fs.s3presto.shaded.com.facebook.presto. > hive.PrestoS3FileSystem.createAmazonS3Client(PrestoS3FileSystem.java:639) > at org.apache.flink.fs.s3presto.shaded.com.facebook.presto. > hive.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:212) > at org.apache.flink.fs.s3presto.S3FileSystemFactory.create( > S3FileSystemFactory.java:132) > at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem( > FileSystem.java:397) > at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:320) > at org.apache.flink.core.fs.Path.getFileSystem(Path.java:293) > at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory.< > init>(FsCheckpointStreamFactory.java:99) > at org.apache.flink.runtime.state.filesystem.FsStateBackend. > createStreamFactory(FsStateBackend.java:277) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > createCheckpointStreamFactory(StreamTask.java:787) > at org.apache.flink.streaming.api.operators.AbstractStreamOperator. > initializeState(AbstractStreamOperator.java:247) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > initializeOperators(StreamTask.java:694) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > initializeState(StreamTask.java:682) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > invoke(StreamTask.java:253) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) > at java.lang.Thread.run(Thread.java:748) > > 01/03/2018 11:21:54 Job execution switched to status FAILING. > java.lang.NoClassDefFoundError: Could not initialize class > org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.internal. > S3ErrorResponseHandler > at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services. > s3.AmazonS3Client.<init>(AmazonS3Client.java:363) > at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services. > s3.AmazonS3Client.<init>(AmazonS3Client.java:542) > at org.apache.flink.fs.s3presto.shaded.com.facebook.presto. > hive.PrestoS3FileSystem.createAmazonS3Client(PrestoS3FileSystem.java:639) > at org.apache.flink.fs.s3presto.shaded.com.facebook.presto. > hive.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:212) > at org.apache.flink.fs.s3presto.S3FileSystemFactory.create( > S3FileSystemFactory.java:132) > at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem( > FileSystem.java:397) > at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:320) > at org.apache.flink.core.fs.Path.getFileSystem(Path.java:293) > at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory.< > init>(FsCheckpointStreamFactory.java:99) > at org.apache.flink.runtime.state.filesystem.FsStateBackend. > createStreamFactory(FsStateBackend.java:277) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > createCheckpointStreamFactory(StreamTask.java:787) > at org.apache.flink.streaming.api.operators.AbstractStreamOperator. > initializeState(AbstractStreamOperator.java:247) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > initializeOperators(StreamTask.java:694) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > initializeState(StreamTask.java:682) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > invoke(StreamTask.java:253) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) > at java.lang.Thread.run(Thread.java:748) > 01/03/2018 11:21:54 Job execution switched to status RESTARTING. > > On Wed, Jan 3, 2018 at 11:33 AM Stephan Ewen <se...@apache.org> wrote: > >> The error is not a the missing Class "S3ErrorResponseHandler", but the >> initialization of that class. >> >> It could be missing classes that are statically referenced by the >> "S3ErrorResponseHandler". >> The ones I see are "XMLInputFactory" (part of Java itself, should always >> be there) and "org.apache.commons.logging.Log" which should also be >> there. >> There may be other classes loaded during initialization. Knowing more >> about the exception would help, as we have not seen that error, yet. >> >> >> On Wed, Jan 3, 2018 at 11:39 AM, Nico Kruber <n...@data-artisans.com> >> wrote: >> >>> Hi Kyle, >>> except for putting the jar into the lib/ folder and setting up >>> credentials, nothing else should be required [1]. >>> >>> The S3ErrorResponseHandler class itself is in the jar, as you can see >>> with >>> jar tf flink-s3-fs-presto-1.4.0.jar | grep >>> org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.internal. >>> S3ErrorResponseHandler >>> >>> Therefore, the cause for this exception would be interesting (as Stephan >>> suggested). >>> >>> >>> Nico >>> >>> [1] >>> https://ci.apache.org/projects/flink/flink-docs- >>> release-1.4/ops/deployment/aws.html#shaded-hadooppresto- >>> s3-file-systems-recommended >>> >>> On 03/01/18 10:22, Stephan Ewen wrote: >>> > Hi Kyle! >>> > >>> > Is there more of the stack trace available, like an original exception >>> > cause? >>> > >>> > Best, >>> > Stephan >>> > >>> > >>> > On Sun, Dec 31, 2017 at 5:10 PM, Kyle Hamlin <hamlin...@gmail.com >>> > <mailto:hamlin...@gmail.com>> wrote: >>> > >>> > Hi, >>> > >>> > When testing Flink 1.4 locally the error below keeps getting >>> thrown. >>> > I've followed the setup by moving the flink-s3-fs-presto.jar from >>> > the opt/ folder to the lib/ folder. Is there something additional I >>> > need to do? >>> > >>> > java.lang.NoClassDefFoundError: Could not initialize class >>> > org.apache.flink.fs.s3presto.shaded.com.amazonaws.services. >>> s3.internal.S3ErrorResponseHandler >>> > at >>> > org.apache.flink.fs.s3presto.shaded.com.amazonaws.services. >>> s3.AmazonS3Client.<init>(AmazonS3Client.java:363) >>> > at >>> > org.apache.flink.fs.s3presto.shaded.com.amazonaws.services. >>> s3.AmazonS3Client.<init>(AmazonS3Client.java:542) >>> > at >>> > org.apache.flink.fs.s3presto.shaded.com.facebook.presto. >>> hive.PrestoS3FileSystem.createAmazonS3Client( >>> PrestoS3FileSystem.java:639) >>> > at >>> > org.apache.flink.fs.s3presto.shaded.com.facebook.presto. >>> hive.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:212) >>> > at >>> > org.apache.flink.fs.s3presto.S3FileSystemFactory.create( >>> S3FileSystemFactory.java:132) >>> > at >>> > org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem( >>> FileSystem.java:397) >>> > at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:320) >>> > at org.apache.flink.core.fs.Path.getFileSystem(Path.java:293) >>> > at >>> > org.apache.flink.runtime.state.filesystem. >>> FsCheckpointStreamFactory.<init>(FsCheckpointStreamFactory.java:99) >>> > at >>> > org.apache.flink.runtime.state.filesystem.FsStateBackend. >>> createStreamFactory(FsStateBackend.java:277) >>> > at >>> > org.apache.flink.streaming.runtime.tasks.StreamTask. >>> createCheckpointStreamFactory(StreamTask.java:787) >>> > at >>> > org.apache.flink.streaming.api.operators.AbstractStreamOperator. >>> initializeState(AbstractStreamOperator.java:247) >>> > at >>> > org.apache.flink.streaming.runtime.tasks.StreamTask. >>> initializeOperators(StreamTask.java:694) >>> > at >>> > org.apache.flink.streaming.runtime.tasks.StreamTask. >>> initializeState(StreamTask.java:682) >>> > at >>> > org.apache.flink.streaming.runtime.tasks.StreamTask. >>> invoke(StreamTask.java:253) >>> > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) >>> > >>> > >>> >>> >>
