[
https://issues.apache.org/jira/browse/BEAM-8830?focusedWorklogId=354394&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354394
]
ASF GitHub Bot logged work on BEAM-8830:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Dec/19 15:46
Start Date: 05/Dec/19 15:46
Worklog Time Spent: 10m
Work Description: echauchot commented on pull request #10293: [BEAM-8830]
Fix flatten in the new spark runner
URL: https://github.com/apache/beam/pull/10293#discussion_r354390280
##########
File path: runners/spark/src/test/resources/log4j-test.properties
##########
@@ -18,7 +18,7 @@
# Set root logger level to ERROR.
# set manually to INFO for debugging purposes.
-log4j.rootLogger=ERROR, testlogger
+log4j.rootLogger=DEBUG, testlogger
Review comment:
well it is just for the runner not for the spark engine. And the runner puts
so few logs (I think only the DAGs).
But anyway, it appears that for some reason, to have the debug logs even in
UTests, it requires to change the production conf log file! Spark has always
been quite difficult to set up with log4j. Maybe it is due to the test file
named log4j-**test**.properties. Anyway I'll put it back to error until we
figure out how to configure it properly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 354394)
Time Spent: 2.5h (was: 2h 20m)
> fix Flatten tests in Spark Structured Streaming runner
> ------------------------------------------------------
>
> Key: BEAM-8830
> URL: https://issues.apache.org/jira/browse/BEAM-8830
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Reporter: Etienne Chauchot
> Assignee: Etienne Chauchot
> Priority: Major
> Labels: structured-streaming
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> 'org.apache.beam.sdk.transforms.FlattenTest.testEmptyFlattenAsSideInput'
> 'org.apache.beam.sdk.transforms.FlattenTest.testFlattenPCollectionsEmpty'
>
> 'org.apache.beam.sdk.transforms.FlattenTest.testFlattenPCollectionsEmptyThenParDo'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
