[jira] [Updated] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large
[ https://issues.apache.org/jira/browse/FLINK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail updated FLINK-25883: Description: In [this line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 315360. This is more than the default value of threading.TIMEOUT_MAX on Windows Python. Due to this, "OverflowError: timeout value is too large" error is produced. Full traceback: {code:java} File "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", line 218, in run while not self._finished.wait(next_call - time.time()): File "C:\Python38\lib\threading.py", line 558, in wait signaled = self._cond.wait(timeout) File "C:\Python38\lib\threading.py", line 306, in wait gotit = waiter.acquire(True, timeout) OverflowError: timeout value is too large{code} was: In [this line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 315360. This is more than the default value of threading.TIMEOUT_MAX on Windows Python. Due to this, "OverflowError: timeout value is too large" error is produced. Full traceback: File "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", line 218, in run while not self._finished.wait(next_call - time.time()): File "C:\Python38\lib\threading.py", line 558, in wait signaled = self._cond.wait(timeout) File "C:\Python38\lib\threading.py", line 306, in wait gotit = waiter.acquire(True, timeout) OverflowError: timeout value is too large > The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large > -- > > Key: FLINK-25883 > URL: https://issues.apache.org/jira/browse/FLINK-25883 > Project: Flink > Issue Type: Bug > Environment: Windows, Python 3.8 >Reporter: Mikhail >Priority: Minor > > In [this > line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], > the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to > 315360. This is more than the default value of threading.TIMEOUT_MAX on > Windows Python. Due to this, "OverflowError: timeout value is too large" > error is produced. > Full traceback: > {code:java} > File > "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", > line 218, in run > while not self._finished.wait(next_call - time.time()): > File "C:\Python38\lib\threading.py", line 558, in wait > signaled = self._cond.wait(timeout) > File "C:\Python38\lib\threading.py", line 306, in wait > gotit = waiter.acquire(True, timeout) > OverflowError: timeout value is too large{code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large
[ https://issues.apache.org/jira/browse/FLINK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail updated FLINK-25883: Description: In [this line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 315360. This is more than the default value of threading.TIMEOUT_MAX on Windows Python, which is 4294967. Due to this, "OverflowError: timeout value is too large" error is produced. Full traceback: {code:java} File "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", line 218, in run while not self._finished.wait(next_call - time.time()): File "C:\Python38\lib\threading.py", line 558, in wait signaled = self._cond.wait(timeout) File "C:\Python38\lib\threading.py", line 306, in wait gotit = waiter.acquire(True, timeout) OverflowError: timeout value is too large{code} was: In [this line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 315360. This is more than the default value of threading.TIMEOUT_MAX on Windows Python. Due to this, "OverflowError: timeout value is too large" error is produced. Full traceback: {code:java} File "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", line 218, in run while not self._finished.wait(next_call - time.time()): File "C:\Python38\lib\threading.py", line 558, in wait signaled = self._cond.wait(timeout) File "C:\Python38\lib\threading.py", line 306, in wait gotit = waiter.acquire(True, timeout) OverflowError: timeout value is too large{code} > The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large > -- > > Key: FLINK-25883 > URL: https://issues.apache.org/jira/browse/FLINK-25883 > Project: Flink > Issue Type: Bug > Environment: Windows, Python 3.8 >Reporter: Mikhail >Priority: Minor > > In [this > line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], > the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to > 315360. This is more than the default value of threading.TIMEOUT_MAX on > Windows Python, which is 4294967. Due to this, "OverflowError: timeout value > is too large" error is produced. > Full traceback: > {code:java} > File > "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", > line 218, in run > while not self._finished.wait(next_call - time.time()): > File "C:\Python38\lib\threading.py", line 558, in wait > signaled = self._cond.wait(timeout) > File "C:\Python38\lib\threading.py", line 306, in wait > gotit = waiter.acquire(True, timeout) > OverflowError: timeout value is too large{code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large
[ https://issues.apache.org/jira/browse/FLINK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail updated FLINK-25883: Description: In [this line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 315360. This is more than the default value of threading.TIMEOUT_MAX on Windows Python. Due to this, "OverflowError: timeout value is too large" error is produced. Full traceback: {code:java} File "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", line 218, in run while not self._finished.wait(next_call - time.time()): File "C:\Python38\lib\threading.py", line 558, in wait signaled = self._cond.wait(timeout) File "C:\Python38\lib\threading.py", line 306, in wait gotit = waiter.acquire(True, timeout) OverflowError: timeout value is too large{code} was: In [this line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 315360. This is more than the default value of threading.TIMEOUT_MAX on Windows Python. Due to this, "OverflowError: timeout value is too large" error is produced. Full traceback: {code:java} File "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", line 218, in run while not self._finished.wait(next_call - time.time()): File "C:\Python38\lib\threading.py", line 558, in wait signaled = self._cond.wait(timeout) File "C:\Python38\lib\threading.py", line 306, in wait gotit = waiter.acquire(True, timeout) OverflowError: timeout value is too large{code} > The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large > -- > > Key: FLINK-25883 > URL: https://issues.apache.org/jira/browse/FLINK-25883 > Project: Flink > Issue Type: Bug > Environment: Windows, Python 3.8 >Reporter: Mikhail >Priority: Minor > > In [this > line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], > the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to > 315360. This is more than the default value of threading.TIMEOUT_MAX on > Windows Python. Due to this, "OverflowError: timeout value is too large" > error is produced. > Full traceback: > {code:java} > File > "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", > line 218, in run > while not self._finished.wait(next_call - time.time()): > File "C:\Python38\lib\threading.py", line 558, in wait > signaled = self._cond.wait(timeout) > File "C:\Python38\lib\threading.py", line 306, in wait > gotit = waiter.acquire(True, timeout) > OverflowError: timeout value is too large{code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large
Mikhail created FLINK-25883: --- Summary: The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large Key: FLINK-25883 URL: https://issues.apache.org/jira/browse/FLINK-25883 Project: Flink Issue Type: Bug Environment: Windows, Python 3.8 Reporter: Mikhail In [this line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30], the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 315360. This is more than the default value of threading.TIMEOUT_MAX on Windows Python. Due to this, "OverflowError: timeout value is too large" error is produced. Full traceback: File "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py", line 218, in run while not self._finished.wait(next_call - time.time()): File "C:\Python38\lib\threading.py", line 558, in wait signaled = self._cond.wait(timeout) File "C:\Python38\lib\threading.py", line 306, in wait gotit = waiter.acquire(True, timeout) OverflowError: timeout value is too large -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-9391) Support UNNEST in Table API
[ https://issues.apache.org/jira/browse/FLINK-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590055#comment-16590055 ] Mikhail commented on FLINK-9391: Hello [~twalthr], As I know, [~ipatina] is on another project. So I'd like to continue Alina's work. I already found out that if we want to select with unnest from table: {code:java} @Test def testUnnest(): Unit = { val env: ExecutionEnvironment = ExecutionEnvironment.getExecutionEnvironment val tEnv = TableEnvironment.getTableEnvironment(env, config) val data = new mutable.MutableList[(Int, Long, Array[String])] data.+=((1, 1L, Array("Hi", "w"))) data.+=((2, 2L, Array("Hello", "k"))) data.+=((3, 2L, Array("Hello world", "x"))) val input = env.fromCollection(Random.shuffle(data)).toTable(tEnv).as('a, 'b, 'c) val unnested = input.select('a, 'b, 'c.unnest()) val actual = unnested.toDataSet[Row].collect() val expected = List("1,Hi", "1,w", "2,Hello", "2,k", "3,Hello world", "3,x").mkString("\n") TestBaseUtils.compareResultAsText(actual.asJava, expected) } {code} Then there will be generated code for a function in DataSetCalc#translateToPlan for processing input data. That code will be compiled and executed. That code will be generated using CommonCalc#generateFunction which will process each element in a Row separately. So final output for a Row in a table will contain single Row. But for our case with UNNEST we need to have another flow. [~twalthr], could you please suggest where to look at? > Support UNNEST in Table API > --- > > Key: FLINK-9391 > URL: https://issues.apache.org/jira/browse/FLINK-9391 > Project: Flink > Issue Type: New Feature > Components: Table API & SQL >Reporter: Timo Walther >Assignee: Alina Ipatina >Priority: Major > > FLINK-6033 introduced the UNNEST function for SQL. We should also add this > function to the Table API to keep the APIs in sync. -- This message was sent by Atlassian JIRA (v7.6.3#76005)