Lucas de Sio Rosa created BEAM-4567:
---------------------------------------
Summary: Can't use mongo connector with Atlas MongoDB
Key: BEAM-4567
URL: https://issues.apache.org/jira/browse/BEAM-4567
Project: Beam
Issue Type: Bug
Components: io-java-mongodb
Affects Versions: 2.4.0
Environment: Google Cloud Dataflow
Reporter: Lucas de Sio Rosa
Assignee: Jean-Baptiste Onofré
I can't use the MongoDB connector with a managed Atlas instance. The current
implementations makes use of splitVector which is a high-privilege function
that cannot be assigned to any user in Atlas.
An open Jira issue for MongoDB suggests using $sample and $bucketAuto to
circunvent this necessity.
Following is the exception thrown (removed some identifiable information):
Exception in thread "main"
org.apache.beam.sdk.Pipeline$PipelineExecutionException:
com.mongodb.MongoCommandException: Command failed with error 13: 'not
authorized on <collection> to execute command \{ splitVector:
"<collection>.<table>", keyPattern: { _id: 1 }, force: false, maxChunkSize: 1
}' on server <server>. The full response is \{ "ok" : 0.0, "errmsg" : "not
authorized on <collection> to execute command { splitVector:
\"<collection>.<table>\", keyPattern: { _id: 1 }, force: false, maxChunkSize: 1
}", "code" : 13, "codeName" : "Unauthorized" }
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at
br.dotz.datalake.ingest.mongodb.MongoDBCollectorPipeline.main(MongoDBCollectorPipeline.java:27)
Caused by: com.mongodb.MongoCommandException: Command failed with error 13:
'not authorized on <collection> to execute command \{ splitVector:
"<collection>.<table>", keyPattern: { _id: 1 }, force: false, maxChunkSize: 1
}' on server <server>. The full response is \{ "ok" : 0.0, "errmsg" : "not
authorized on <collection> to execute command { splitVector:
\"<collection>.<table>\", keyPattern: { _id: 1 }, force: false, maxChunkSize: 1
}", "code" : 13, "codeName" : "Unauthorized" }
at
com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)
at
com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159)
at
com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286)
at
com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173)
at
com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215)
at
com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:186)
at
com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:178)
at
com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:91)
at
com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:84)
at
com.mongodb.operation.CommandReadOperation.execute(CommandReadOperation.java:55)
at com.mongodb.Mongo.execute(Mongo.java:772)
at com.mongodb.Mongo$2.execute(Mongo.java:759)
at com.mongodb.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:130)
at com.mongodb.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:124)
at com.mongodb.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:114)
at
org.apache.beam.sdk.io.mongodb.MongoDbIO$BoundedMongoDbSource.split(MongoDbIO.java:332)
at
org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:210)
at
org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:87)
at
org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:62)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
