[jira] [Updated] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large

2022-01-30 Thread Mikhail (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail updated FLINK-25883:

Description: 
In [this 
line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
 the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
315360. This is more than the default value of threading.TIMEOUT_MAX on 
Windows Python. Due to this, "OverflowError: timeout value is too large" error 
is produced.

Full traceback:
{code:java}
File 
"G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
 line 218, in run
while not self._finished.wait(next_call - time.time()):
File "C:\Python38\lib\threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "C:\Python38\lib\threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
OverflowError: timeout value is too large{code}

  was:
In [this 
line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
 the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
315360. This is more than the default value of threading.TIMEOUT_MAX on 
Windows Python. Due to this, "OverflowError: timeout value is too large" error 
is produced.

Full traceback:
  File 
"G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
 line 218, in run
while not self._finished.wait(next_call - time.time()):
  File "C:\Python38\lib\threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
  File "C:\Python38\lib\threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
OverflowError: timeout value is too large


> The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large 
> --
>
> Key: FLINK-25883
> URL: https://issues.apache.org/jira/browse/FLINK-25883
> Project: Flink
>  Issue Type: Bug
> Environment: Windows, Python 3.8
>Reporter: Mikhail
>Priority: Minor
>
> In [this 
> line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
>  the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
> 315360. This is more than the default value of threading.TIMEOUT_MAX on 
> Windows Python. Due to this, "OverflowError: timeout value is too large" 
> error is produced.
> Full traceback:
> {code:java}
> File 
> "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
>  line 218, in run
> while not self._finished.wait(next_call - time.time()):
> File "C:\Python38\lib\threading.py", line 558, in wait
> signaled = self._cond.wait(timeout)
> File "C:\Python38\lib\threading.py", line 306, in wait
> gotit = waiter.acquire(True, timeout)
> OverflowError: timeout value is too large{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large

2022-01-30 Thread Mikhail (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail updated FLINK-25883:

Description: 
In [this 
line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
 the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
315360. This is more than the default value of threading.TIMEOUT_MAX on 
Windows Python, which is 4294967. Due to this, "OverflowError: timeout value is 
too large" error is produced.

Full traceback:
{code:java}
 File 
"G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
 line 218, in run
  while not self._finished.wait(next_call - time.time()):
 File "C:\Python38\lib\threading.py", line 558, in wait
  signaled = self._cond.wait(timeout)
 File "C:\Python38\lib\threading.py", line 306, in wait
  gotit = waiter.acquire(True, timeout)
OverflowError: timeout value is too large{code}

  was:
In [this 
line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
 the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
315360. This is more than the default value of threading.TIMEOUT_MAX on 
Windows Python. Due to this, "OverflowError: timeout value is too large" error 
is produced.

Full traceback:
{code:java}
 File 
"G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
 line 218, in run
  while not self._finished.wait(next_call - time.time()):
 File "C:\Python38\lib\threading.py", line 558, in wait
  signaled = self._cond.wait(timeout)
 File "C:\Python38\lib\threading.py", line 306, in wait
  gotit = waiter.acquire(True, timeout)
OverflowError: timeout value is too large{code}


> The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large 
> --
>
> Key: FLINK-25883
> URL: https://issues.apache.org/jira/browse/FLINK-25883
> Project: Flink
>  Issue Type: Bug
> Environment: Windows, Python 3.8
>Reporter: Mikhail
>Priority: Minor
>
> In [this 
> line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
>  the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
> 315360. This is more than the default value of threading.TIMEOUT_MAX on 
> Windows Python, which is 4294967. Due to this, "OverflowError: timeout value 
> is too large" error is produced.
> Full traceback:
> {code:java}
>  File 
> "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
>  line 218, in run
>   while not self._finished.wait(next_call - time.time()):
>  File "C:\Python38\lib\threading.py", line 558, in wait
>   signaled = self._cond.wait(timeout)
>  File "C:\Python38\lib\threading.py", line 306, in wait
>   gotit = waiter.acquire(True, timeout)
> OverflowError: timeout value is too large{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large

2022-01-30 Thread Mikhail (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail updated FLINK-25883:

Description: 
In [this 
line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
 the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
315360. This is more than the default value of threading.TIMEOUT_MAX on 
Windows Python. Due to this, "OverflowError: timeout value is too large" error 
is produced.

Full traceback:
{code:java}
 File 
"G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
 line 218, in run
  while not self._finished.wait(next_call - time.time()):
 File "C:\Python38\lib\threading.py", line 558, in wait
  signaled = self._cond.wait(timeout)
 File "C:\Python38\lib\threading.py", line 306, in wait
  gotit = waiter.acquire(True, timeout)
OverflowError: timeout value is too large{code}

  was:
In [this 
line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
 the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
315360. This is more than the default value of threading.TIMEOUT_MAX on 
Windows Python. Due to this, "OverflowError: timeout value is too large" error 
is produced.

Full traceback:
{code:java}
File 
"G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
 line 218, in run
while not self._finished.wait(next_call - time.time()):
File "C:\Python38\lib\threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "C:\Python38\lib\threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
OverflowError: timeout value is too large{code}


> The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large 
> --
>
> Key: FLINK-25883
> URL: https://issues.apache.org/jira/browse/FLINK-25883
> Project: Flink
>  Issue Type: Bug
> Environment: Windows, Python 3.8
>Reporter: Mikhail
>Priority: Minor
>
> In [this 
> line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
>  the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
> 315360. This is more than the default value of threading.TIMEOUT_MAX on 
> Windows Python. Due to this, "OverflowError: timeout value is too large" 
> error is produced.
> Full traceback:
> {code:java}
>  File 
> "G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
>  line 218, in run
>   while not self._finished.wait(next_call - time.time()):
>  File "C:\Python38\lib\threading.py", line 558, in wait
>   signaled = self._cond.wait(timeout)
>  File "C:\Python38\lib\threading.py", line 306, in wait
>   gotit = waiter.acquire(True, timeout)
> OverflowError: timeout value is too large{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (FLINK-25883) The value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large

2022-01-30 Thread Mikhail (Jira) 
Mikhail created FLINK-25883:
---

 Summary: The value of 
DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is too large 
 Key: FLINK-25883
 URL: https://issues.apache.org/jira/browse/FLINK-25883
 Project: Flink
  Issue Type: Bug
 Environment: Windows, Python 3.8
Reporter: Mikhail


In [this 
line|https://github.com/apache/flink/blob/fb38c99a38c63ba8801e765887f955522072615a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py#L30],
 the value of DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S is set to 
315360. This is more than the default value of threading.TIMEOUT_MAX on 
Windows Python. Due to this, "OverflowError: timeout value is too large" error 
is produced.

Full traceback:
  File 
"G:\PycharmProjects\PyFlink\venv_from_scratch\lib\site-packages\apache_beam\runners\worker\data_plane.py",
 line 218, in run
while not self._finished.wait(next_call - time.time()):
  File "C:\Python38\lib\threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
  File "C:\Python38\lib\threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
OverflowError: timeout value is too large



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (FLINK-9391) Support UNNEST in Table API

2018-08-23 Thread Mikhail (JIRA) 


[ 
https://issues.apache.org/jira/browse/FLINK-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590055#comment-16590055
 ] 

Mikhail commented on FLINK-9391:


Hello [~twalthr],

As I know, [~ipatina] is on another project. So I'd like to continue Alina's 
work.

I already found out that if we want to select with unnest from table:
{code:java}
@Test
def testUnnest(): Unit = {
val env: ExecutionEnvironment = ExecutionEnvironment.getExecutionEnvironment
val tEnv = TableEnvironment.getTableEnvironment(env, config)

val data = new mutable.MutableList[(Int, Long, Array[String])]
data.+=((1, 1L, Array("Hi", "w")))
data.+=((2, 2L, Array("Hello", "k")))
data.+=((3, 2L, Array("Hello world", "x")))
val input = env.fromCollection(Random.shuffle(data)).toTable(tEnv).as('a, 'b, 
'c)

val unnested = input.select('a, 'b, 'c.unnest())

val actual = unnested.toDataSet[Row].collect()

val expected = List("1,Hi", "1,w", "2,Hello", "2,k", "3,Hello world", 
"3,x").mkString("\n")
TestBaseUtils.compareResultAsText(actual.asJava, expected)
}
{code}
Then there will be generated code for a function in DataSetCalc#translateToPlan 
for processing input data. That code will be compiled and executed.

That code will be generated using CommonCalc#generateFunction which will 
process each element in a Row separately.

So final output for a Row in a table will contain single Row.

But for our case with UNNEST we need to have another flow.

[~twalthr], could you please suggest where to look at?

> Support UNNEST in Table API
> ---
>
> Key: FLINK-9391
> URL: https://issues.apache.org/jira/browse/FLINK-9391
> Project: Flink
>  Issue Type: New Feature
>  Components: Table API  SQL
>Reporter: Timo Walther
>Assignee: Alina Ipatina
>Priority: Major
>
> FLINK-6033 introduced the UNNEST function for SQL. We should also add this 
> function to the Table API to keep the APIs in sync. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)