[ https://issues.apache.org/jira/browse/BEAM-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739585#comment-16739585 ]
Chamikara Jayalath commented on BEAM-5966: ------------------------------------------ Was your question resolved or is there something more to address here ? > When working "LiteralGqlQuery", Error: Query cannot have any sort orders. > ------------------------------------------------------------------------- > > Key: BEAM-5966 > URL: https://issues.apache.org/jira/browse/BEAM-5966 > Project: Beam > Issue Type: Bug > Components: io-java-gcp > Affects Versions: 2.8.0 > Reporter: Murat ODUNC > Assignee: Chamikara Jayalath > Priority: Major > > Hello.. > I have used Apache Beam pipelines on my project. And I have a problem to read > data from GCP DataStore. > > My First question is "How to create "Query" object" . I'am sharing a sample > of regarding JavaDoc. > [https://beam.apache.org/releases/javadoc/2.8.0/] > {code:java} > Query query = ...; // I dont know any idea how configure the object > String projectId = "..."; > Pipeline p = Pipeline.create(options); > PCollection<Entity> entities = p.apply( > DatastoreIO.v1().read() > .withProjectId(projectId) > .withQuery(query)); > {code} > My second question is how to set "sort oder" of the query? > I tried to 'LiteralGqlQuery' to read data but DataStore IO Reader failed by > follow error trace.. > {noformat} > java.lang.IllegalArgumentException: Query cannot have any sort orders. at > com.google.datastore.v1.client.QuerySplitterImpl.validateQuery(QuerySplitterImpl.java:128) > ~[datastore-v1-proto-client-1.6.0.jar:na] at > com.google.datastore.v1.client.QuerySplitterImpl.getSplits(QuerySplitterImpl.java:69) > ~[datastore-v1-proto-client-1.6.0.jar:na] at > org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read.splitQuery(DatastoreV1.java:454) > ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at > org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read.access$100(DatastoreV1.java:264) > ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at > org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read$SplitQueryFn.processElement(DatastoreV1.java:813) > ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at > org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read$SplitQueryFn$DoFnInvoker.invokeProcessElement(Unknown > Source) [na:na] at > org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:275) > [beam-runners-direct-java-2.8.0.jar:na] at > org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:240) > [beam-runners-direct-java-2.8.0.jar:na] at > org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows(SimplePushbackSideInputDoFnRunner.java:78) > [beam-runners-direct-java-2.8.0.jar:na] > at > org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:207) > [beam-runners-direct-java-2.8.0.jar:na] at > org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:55) > [beam-runners-direct-java-2.8.0.jar:na] at > org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160) > [beam-runners-direct-java-2.8.0.jar:na] at > org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124) > [beam-runners-direct-java-2.8.0.jar:na] at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_162] at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [na:1.8.0_162] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [na:1.8.0_162] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [na:1.8.0_162] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162] > {noformat} > My code is here > > {code:java} > PipelineOptions options = PipelineOptionsFactory.create(); > options.setRunner(DirectRunner.class); String gqlQuery = > String.format("SELECT * FROM task " + "WHERE createdAt > DATETIME('%s') "+ > "AND createdAt < DATETIME('%s') ORDER BY createdAt ASC", > "2018-11-03T00:00:00Z", "2018-11-03T23:59:59Z" ); log.info("GQL: " + > gqlQuery); String projectId = "foo-bar"; String ns = > "prod-01ce205a-22ff-4bab-b133-926bdc54c8b3"; > Pipeline p = Pipeline.create(options); > PCollection<String> entities = p.apply( DatastoreIO.v1().read() > .withProjectId(projectId).withLiteralGqlQuery(gqlQuery) .withQuery(query) > .withNamespace(ns)) .apply("Debug", ParDo.of(new DoFn<Entity, String>(){ > @ProcessElement > public void processElement(ProcessContext c) { > > log.info(c.element().getProperties().get("foo").getStringValue()); > } > })); > p.run().waitUntilFinish(); > {code} > ``` > -- This message was sent by Atlassian JIRA (v7.6.3#76005)