[jira] [Created] (AIRFLOW-567) Update links in README
Wesley created AIRFLOW-567: -- Summary: Update links in README Key: AIRFLOW-567 URL: https://issues.apache.org/jira/browse/AIRFLOW-567 Project: Apache Airflow Issue Type: Wish Components: docs Reporter: Wesley Priority: Trivial Hello guys, Could you please update the Easy Taxi links in the readme? {code:none} [Easy Taxi](http://www.easytaxi.com/) [[@caique-lima](https://github.com/caique-lima), [@WesleyBatista](https://github.com/WesleyBatista)] {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-567) Update links in README
[ https://issues.apache.org/jira/browse/AIRFLOW-567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley updated AIRFLOW-567: --- Labels: documentation (was: ) > Update links in README > -- > > Key: AIRFLOW-567 > URL: https://issues.apache.org/jira/browse/AIRFLOW-567 > Project: Apache Airflow > Issue Type: Wish > Components: docs >Reporter: Wesley >Priority: Trivial > Labels: documentation > > Hello guys, > Could you please update the Easy Taxi links in the readme? > {code:none} > [Easy Taxi](http://www.easytaxi.com/) > [[@caique-lima](https://github.com/caique-lima), > [@WesleyBatista](https://github.com/WesleyBatista)] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-567) Update links in README
[ https://issues.apache.org/jira/browse/AIRFLOW-567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley updated AIRFLOW-567: --- Description: Hello guys, Could you please update the Easy Taxi links in the readme? Related PR: https://github.com/apache/incubator-airflow/pull/1835 was: Hello guys, Could you please update the Easy Taxi links in the readme? {code:none} [Easy Taxi](http://www.easytaxi.com/) [[@caique-lima](https://github.com/caique-lima), [@WesleyBatista](https://github.com/WesleyBatista)] {code} > Update links in README > -- > > Key: AIRFLOW-567 > URL: https://issues.apache.org/jira/browse/AIRFLOW-567 > Project: Apache Airflow > Issue Type: Wish > Components: docs >Reporter: Wesley >Priority: Trivial > Labels: documentation > > Hello guys, > Could you please update the Easy Taxi links in the readme? > Related PR: https://github.com/apache/incubator-airflow/pull/1835 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-3330) Update the Airflow users list
Wesley created AIRFLOW-3330: --- Summary: Update the Airflow users list Key: AIRFLOW-3330 URL: https://issues.apache.org/jira/browse/AIRFLOW-3330 Project: Apache Airflow Issue Type: Wish Components: Documentation Reporter: Wesley -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3330) Update the Airflow users list
[ https://issues.apache.org/jira/browse/AIRFLOW-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley updated AIRFLOW-3330: Description: PR: https://github.com/apache/incubator-airflow/pull/4176 > Update the Airflow users list > - > > Key: AIRFLOW-3330 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3330 > Project: Apache Airflow > Issue Type: Wish > Components: Documentation >Reporter: Wesley >Priority: Major > > PR: https://github.com/apache/incubator-airflow/pull/4176 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2843) ExternalTaskSensor: Add option to cease waiting immediately if the external DAG/task doesn't exist
[ https://issues.apache.org/jira/browse/AIRFLOW-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929083#comment-16929083 ] Wesley commented on AIRFLOW-2843: - Using the operator stumbled upon the case where the `external_dag_id` and `external_task_id` exist but the DAG is disabled, so the ExternalTaskSensor task instance keep polling for a DagRun that does not exist. Perhaps the `check_existence` should check DagRun instead? or the name should be more specific like `check_dag_existence`/ `check_task_existence`? What you guys think about this use case? > ExternalTaskSensor: Add option to cease waiting immediately if the external > DAG/task doesn't exist > -- > > Key: AIRFLOW-2843 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2843 > Project: Apache Airflow > Issue Type: Improvement > Components: operators >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > Fix For: 1.10.3 > > > h2. Background > *ExternalTaskSensor* will keep waiting (given restrictions of retries, > poke_interval, etc), even if the external DAG/task specified doesn't exist at > all. In some cases, this waiting may still make sense as new DAG may backfill. > But it may be good to provide an option to cease waiting immediately if the > external DAG/task specified doesn't exist. > h2. Proposal > Provide an argument "check_existence". Set to *True* to check if the external > DAG/task exists, and immediately cease waiting if the external DAG/task does > not exist. > The default value is set to *False* (no check or ceasing will happen) so it > will not affect any existing DAGs or user expectation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (AIRFLOW-2843) ExternalTaskSensor: Add option to cease waiting immediately if the external DAG/task doesn't exist
[ https://issues.apache.org/jira/browse/AIRFLOW-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929083#comment-16929083 ] Wesley edited comment on AIRFLOW-2843 at 9/13/19 10:08 AM: --- Using the operator stumbled upon the case where the `external_dag_id` and `external_task_id` exist but the DAG is disabled, so the ExternalTaskSensor task instance keep poking for a DagRun that does not exist. Perhaps the `check_existence` should check DagRun instead? or the name should be more specific like `check_dag_existence`/ `check_task_existence`? What you guys think about this use case? was (Author: wesleybatista): Using the operator stumbled upon the case where the `external_dag_id` and `external_task_id` exist but the DAG is disabled, so the ExternalTaskSensor task instance keep polling for a DagRun that does not exist. Perhaps the `check_existence` should check DagRun instead? or the name should be more specific like `check_dag_existence`/ `check_task_existence`? What you guys think about this use case? > ExternalTaskSensor: Add option to cease waiting immediately if the external > DAG/task doesn't exist > -- > > Key: AIRFLOW-2843 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2843 > Project: Apache Airflow > Issue Type: Improvement > Components: operators >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > Fix For: 1.10.3 > > > h2. Background > *ExternalTaskSensor* will keep waiting (given restrictions of retries, > poke_interval, etc), even if the external DAG/task specified doesn't exist at > all. In some cases, this waiting may still make sense as new DAG may backfill. > But it may be good to provide an option to cease waiting immediately if the > external DAG/task specified doesn't exist. > h2. Proposal > Provide an argument "check_existence". Set to *True* to check if the external > DAG/task exists, and immediately cease waiting if the external DAG/task does > not exist. > The default value is set to *False* (no check or ceasing will happen) so it > will not affect any existing DAGs or user expectation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (AMQ-7284) Newly restarted consumer becomes exclusive consumer of ActiveMQ messages for new message groups
Wesley created AMQ-7284: --- Summary: Newly restarted consumer becomes exclusive consumer of ActiveMQ messages for new message groups Key: AMQ-7284 URL: https://issues.apache.org/jira/browse/AMQ-7284 Project: ActiveMQ Issue Type: Bug Reporter: Wesley Suppose there are 2 single threaded consumers A and B (2 separate processes) listening to messages on a queue. All messages sent to this group have a group ID (JMSXGroupID). ActiveMQ has selected A as the consumer for message groups G1, G2, G3, ... G500. Now, if A goes down and end-group messages ("JMSXGroupSeq" = -1) arrive for these groups, then consumer B consumes them and these groups get permanently attached to consumer B even though the groups have ended. I checked this using ./activemq (formerly activemq-admin) query -QQueue=*test.queue* --view MessageGroups Now there is no way these message groups will get purged unless a new end-group message arrives with the same group ID or 'removeAllMessageGroups' is invoked through JMX. Now if A comes back up again and normal behavior continues, A will get all of the new groups unless there are at least 500 active groups within A that are not yet terminated. It's only after A has at least 500 active groups that B will get a chance to be assigned message groups, but the lower bound for B remains 500. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (AMQ-7284) Newly restarted consumer becomes exclusive consumer of ActiveMQ messages for new message groups
[ https://issues.apache.org/jira/browse/AMQ-7284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley updated AMQ-7284: Affects Version/s: 5.15.9 > Newly restarted consumer becomes exclusive consumer of ActiveMQ messages for > new message groups > --- > > Key: AMQ-7284 > URL: https://issues.apache.org/jira/browse/AMQ-7284 > Project: ActiveMQ > Issue Type: Bug >Affects Versions: 5.15.9 >Reporter: Wesley >Priority: Major > > Suppose there are 2 single threaded consumers A and B (2 separate processes) > listening to messages on a queue. All messages sent to this group have a > group ID (JMSXGroupID). > ActiveMQ has selected A as the consumer for message groups G1, G2, G3, ... > G500. Now, if A goes down and end-group messages ("JMSXGroupSeq" = -1) arrive > for these groups, then consumer B consumes them and these groups get > permanently attached to consumer B even though the groups have ended. > I checked this using ./activemq (formerly activemq-admin) query > -QQueue=*test.queue* --view MessageGroups > Now there is no way these message groups will get purged unless a new > end-group message arrives with the same group ID or 'removeAllMessageGroups' > is invoked through JMX. > Now if A comes back up again and normal behavior continues, A will get all of > the new groups unless there are at least 500 active groups within A that are > not yet terminated. It's only after A has at least 500 active groups that B > will get a chance to be assigned message groups, but the lower bound for B > remains 500. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] Updated: (GERONIMO-2515) load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10
[ http://issues.apache.org/jira/browse/GERONIMO-2515?page=all ] K Wesley updated GERONIMO-2515: --- Priority: Blocker (was: Major) > load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10 > -- > > Key: GERONIMO-2515 > URL: http://issues.apache.org/jira/browse/GERONIMO-2515 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) > Components: startup/shutdown >Affects Versions: 1.1.1 > Environment: Solaris 10 , JDK 1.5.0_01 >Reporter: K Wesley >Priority: Blocker > > $ java -jar bin/server.jar > Booting Geronimo Kernel (in Java 1.5.0_01)... > Starting Geronimo Application Server v1.1.1 > [*-] 0% 2s Startup failed > org.apache.geronimo.kernel.config.LifecycleException: load of > geronimo/rmi-naming/1.1.1/car failed > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.config.EditableConfigurationManager$$EnhancerByCGLIB$$7c7a552c.loadConfiguration() > at org.apache.geronimo.system.main.Daemon.doStartup(Daemon.java:294) > at org.apache.geronimo.system.main.Daemon.(Daemon.java:74) > at org.apache.geronimo.system.main.Daemon.main(Daemon.java:377) > Caused by: org.apache.geronimo.kernel.repository.MissingDependencyException: > Unable to resolve dependency > org.apache.geronimo.specs/geronimo-activation_1.0.2_spec//jar > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:119) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:98) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver$$FastClassByCGLIB$$e847b746.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.repository.ArtifactResolver$$EnhancerByCGLIB$$c9e2351.resolveInClassLoader() > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.resolveParentIds(SimpleConfigurationManager.java:466) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadDepthFirst(SimpleConfigurationManager.java:425) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:291) > ... 15 more > Server shutdown begun > Server shutdown completed -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (GERONIMO-2515) load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10
[ http://issues.apache.org/jira/browse/GERONIMO-2515?page=comments#action_12446235 ] K Wesley commented on GERONIMO-2515: I will install 1.4.2 today. The default on solaris 10 is 1.5 > load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10 > -- > > Key: GERONIMO-2515 > URL: http://issues.apache.org/jira/browse/GERONIMO-2515 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) > Components: startup/shutdown >Affects Versions: 1.1.1 > Environment: Solaris 10 , JDK 1.5.0_01 >Reporter: K Wesley >Priority: Blocker > > $ java -jar bin/server.jar > Booting Geronimo Kernel (in Java 1.5.0_01)... > Starting Geronimo Application Server v1.1.1 > [*-] 0% 2s Startup failed > org.apache.geronimo.kernel.config.LifecycleException: load of > geronimo/rmi-naming/1.1.1/car failed > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.config.EditableConfigurationManager$$EnhancerByCGLIB$$7c7a552c.loadConfiguration() > at org.apache.geronimo.system.main.Daemon.doStartup(Daemon.java:294) > at org.apache.geronimo.system.main.Daemon.(Daemon.java:74) > at org.apache.geronimo.system.main.Daemon.main(Daemon.java:377) > Caused by: org.apache.geronimo.kernel.repository.MissingDependencyException: > Unable to resolve dependency > org.apache.geronimo.specs/geronimo-activation_1.0.2_spec//jar > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:119) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:98) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver$$FastClassByCGLIB$$e847b746.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.repository.ArtifactResolver$$EnhancerByCGLIB$$c9e2351.resolveInClassLoader() > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.resolveParentIds(SimpleConfigurationManager.java:466) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadDepthFirst(SimpleConfigurationManager.java:425) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:291) > ... 15 more > Server shutdown begun > Server shutdown completed -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (GERONIMO-2515) load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10
[ http://issues.apache.org/jira/browse/GERONIMO-2515?page=comments#action_12446262 ] K Wesley commented on GERONIMO-2515: same error: $ java -jar server.jar Booting Geronimo Kernel (in Java 1.4.2_12)... Starting Geronimo Application Server v1.1.1 [*-] 0% 2s Startup failed org.apache.geronimo.kernel.config.LifecycleException: load of geronimo/rmi-naming/1.1.1/car failed at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) at org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) at org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() > load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10 > -- > > Key: GERONIMO-2515 > URL: http://issues.apache.org/jira/browse/GERONIMO-2515 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) > Components: startup/shutdown >Affects Versions: 1.1.1 > Environment: Solaris 10 , JDK 1.5.0_01 >Reporter: K Wesley >Priority: Blocker > > $ java -jar bin/server.jar > Booting Geronimo Kernel (in Java 1.5.0_01)... > Starting Geronimo Application Server v1.1.1 > [*-] 0% 2s Startup failed > org.apache.geronimo.kernel.config.LifecycleException: load of > geronimo/rmi-naming/1.1.1/car failed > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.config.EditableConfigurationManager$$EnhancerByCGLIB$$7c7a552c.loadConfiguration() > at org.apache.geronimo.system.main.Daemon.doStartup(Daemon.java:294) > at org.apache.geronimo.system.main.Daemon.(Daemon.java:74) > at org.apache.geronimo.system.main.Daemon.main(Daemon.java:377) > Caused by: org.apache.geronimo.kernel.repository.MissingDependencyException: > Unable to resolve dependency > org.apache.geronimo.specs/geronimo-activation_1.0.2_spec//jar > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:119) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:98) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver$$FastClassByCGLIB$$e847b746.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.repository.ArtifactResolver$$EnhancerByCGLIB$$c9e2351.resolveInClassLoader() > at > org.apache.geronimo.kernel.config.SimpleConfi
[jira] Updated: (GERONIMO-2515) load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10
[ http://issues.apache.org/jira/browse/GERONIMO-2515?page=all ] K Wesley updated GERONIMO-2515: --- Description: $ java -jar server.jar Booting Geronimo Kernel (in Java 1.4.2_12)... Starting Geronimo Application Server v1.1.1 [*-] 0% 2s Startup failed org.apache.geronimo.kernel.config.LifecycleException: load of geronimo/rmi-naming/1.1.1/car failed at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) at org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) at org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) at org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) at org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) at org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) at org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) at org.apache.geronimo.kernel.config.EditableConfigurationManager$$EnhancerByCGLIB$$fe617d34.loadConfiguration() at org.apache.geronimo.system.main.Daemon.doStartup(Daemon.java:294) at org.apache.geronimo.system.main.Daemon.(Daemon.java:74) at org.apache.geronimo.system.main.Daemon.main(Daemon.java:377) Caused by: org.apache.geronimo.kernel.repository.MissingDependencyException: Unable to resolve dependency org.apache.geronimo.specs/geronimo-activation_1.0.2_spec//jar at org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:119) at org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:98) at org.apache.geronimo.kernel.repository.DefaultArtifactResolver$$FastClassByCGLIB$$e847b746.invoke() at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) at org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) at org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) at org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) at org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) at org.apache.geronimo.kernel.repository.ArtifactResolver$$EnhancerByCGLIB$$8e854b59.resolveInClassLoader() at org.apache.geronimo.kernel.config.SimpleConfigurationManager.resolveParentIds(SimpleConfigurationManager.java:466) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadDepthFirst(SimpleConfigurationManager.java:425) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:291) ... 15 more Server shutdown begun Server shutdown completed was: $ java -jar bin/server.jar Booting Geronimo Kernel (in Java 1.5.0_01)... Starting Geronimo Application Server v1.1.1 [*-] 0% 2s Startup failed org.apache.geronimo.kernel.config.LifecycleException: load of geronimo/rmi-naming/1.1.1/car failed at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) at org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) at org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) at org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) at org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) at
[jira] Commented: (GERONIMO-2515) load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10
[ http://issues.apache.org/jira/browse/GERONIMO-2515?page=comments#action_12449373 ] K Wesley commented on GERONIMO-2515: Yes, I originally tested this on 1.5 and then tested on 1.4.2, since this is the officially supported release. I just think that this does not work on solaris anywhere sparc or x86. Is there a chance that this will be fixed soon? We are using geronimo in our upcoming release and solaris is one of supported platforms. > load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10 > -- > > Key: GERONIMO-2515 > URL: http://issues.apache.org/jira/browse/GERONIMO-2515 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) > Components: startup/shutdown >Affects Versions: 1.1.1 > Environment: Solaris 10 , JDK 1.4.2 >Reporter: K Wesley >Priority: Blocker > > $ java -jar server.jar > Booting Geronimo Kernel (in Java 1.4.2_12)... > Starting Geronimo Application Server v1.1.1 > [*-] 0% 2s Startup failed > org.apache.geronimo.kernel.config.LifecycleException: load of > geronimo/rmi-naming/1.1.1/car failed > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.config.EditableConfigurationManager$$EnhancerByCGLIB$$fe617d34.loadConfiguration() > at org.apache.geronimo.system.main.Daemon.doStartup(Daemon.java:294) > at org.apache.geronimo.system.main.Daemon.(Daemon.java:74) > at org.apache.geronimo.system.main.Daemon.main(Daemon.java:377) > Caused by: org.apache.geronimo.kernel.repository.MissingDependencyException: > Unable to resolve dependency > org.apache.geronimo.specs/geronimo-activation_1.0.2_spec//jar > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:119) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:98) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver$$FastClassByCGLIB$$e847b746.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.repository.ArtifactResolver$$EnhancerByCGLIB$$8e854b59.resolveInClassLoader() > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.resolveParentIds(SimpleConfigurationManager.java:466) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadDepthFirst(SimpleConfigurationManager.java:425) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:291) > ... 15 more > Server shutdown begun > Server shutdown completed -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.j
[jira] Commented: (GERONIMO-2515) load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10
[ https://issues.apache.org/jira/browse/GERONIMO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499859 ] K Wesley commented on GERONIMO-2515: How does one get the builds that you are mentioning? I am using the one available on the download page from apache geronimo here: http://geronimo.apache.org/downloads.html > load of geronimo/rmi-naming/1.1.1/car failed on Solaris 10 > -- > > Key: GERONIMO-2515 > URL: https://issues.apache.org/jira/browse/GERONIMO-2515 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) > Components: startup/shutdown >Affects Versions: 1.1.1 > Environment: Solaris 10 , JDK 1.4.2 >Reporter: K Wesley >Priority: Blocker > > $ java -jar server.jar > Booting Geronimo Kernel (in Java 1.4.2_12)... > Starting Geronimo Application Server v1.1.1 > [*-] 0% 2s Startup failed > org.apache.geronimo.kernel.config.LifecycleException: load of > geronimo/rmi-naming/1.1.1/car failed > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:294) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:275) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:250) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager.loadConfiguration(KernelConfigurationManager.java:112) > at > org.apache.geronimo.kernel.config.KernelConfigurationManager$$FastClassByCGLIB$$b117102f.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.config.EditableConfigurationManager$$EnhancerByCGLIB$$fe617d34.loadConfiguration() > at org.apache.geronimo.system.main.Daemon.doStartup(Daemon.java:294) > at org.apache.geronimo.system.main.Daemon.(Daemon.java:74) > at org.apache.geronimo.system.main.Daemon.main(Daemon.java:377) > Caused by: org.apache.geronimo.kernel.repository.MissingDependencyException: > Unable to resolve dependency > org.apache.geronimo.specs/geronimo-activation_1.0.2_spec//jar > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:119) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver.resolveInClassLoader(DefaultArtifactResolver.java:98) > at > org.apache.geronimo.kernel.repository.DefaultArtifactResolver$$FastClassByCGLIB$$e847b746.invoke() > at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) > at > org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke(FastMethodInvoker.java:38) > at > org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) > at > org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:817) > at > org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) > at > org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) > at > org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) > at > org.apache.geronimo.kernel.repository.ArtifactResolver$$EnhancerByCGLIB$$8e854b59.resolveInClassLoader() > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.resolveParentIds(SimpleConfigurationManager.java:466) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadDepthFirst(SimpleConfigurationManager.java:425) > at > org.apache.geronimo.kernel.config.SimpleConfigurationManager.loadConfiguration(SimpleConfigurationManager.java:291) > ... 15 more > Server shutdown begun > Server shutdown completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (OFBIZ-2052) Convert Mypage into the Myportal component using the new portal functions
[ https://issues.apache.org/jira/browse/OFBIZ-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648994#action_12648994 ] Sven Wesley commented on OFBIZ-2052: Portlets are very nice. Is there any plan to implement the portlet functionality in other components than the MyPortal/Dashboard? Another question, is this following the standard for portlets (JSR-286)? It would be such a nice feature to have portlets in the customer front end. > Convert Mypage into the Myportal component using the new portal functions > - > > Key: OFBIZ-2052 > URL: https://issues.apache.org/jira/browse/OFBIZ-2052 > Project: OFBiz > Issue Type: New Feature > Components: specialpurpose/mypage >Affects Versions: SVN trunk >Reporter: Hans Bakker > Fix For: SVN trunk > > Attachments: portal.diff > > > Now the portal functions is in the trunk, i had a more closer look how to use > this in the mypage component. > The result is that I would like to propose a new component call "My Portal" > which will replace the "My Page" component. This will give me the opportunity > to slowly (in 3-4 weeks) make the conversion. > Attached is a diff file when applied will create the first version of the new > component showing the principle, how the portal functions could be used. > When installed it will create a new tab called "my Portal" initially only > with the default page. > When you click on "preferences" you see a list of portal pages, and a form to > add new ones. > If a new one is added, also a new button will show for that page. > When you click on the new page Id, the configuration screen appears and a > portlet can be added and when there are more they can be shifted around. > When no objections then i will commit this version and will extend it and > finally will remove the mypage component. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] (MSHARED-288) Allow Maven Archiver to not include pom.xml and/or pom.properties files
Justin Wesley created MSHARED-288: - Summary: Allow Maven Archiver to not include pom.xml and/or pom.properties files Key: MSHARED-288 URL: https://jira.codehaus.org/browse/MSHARED-288 Project: Maven Shared Components Issue Type: New Feature Components: maven-archiver Affects Versions: maven-archiver-2.4 Reporter: Justin Wesley Priority: Minor Currently, when configuring the archive for a package, the developer can only include both pom.properties and pom.xml files, or neither of them. They can't pick to only include one or the other. Some projects may want to ability to determine programatically basic information of the jar they have by using the properties file, but not include the pom.xml file as it may contain information that may expose information about the build that could be specific to a build machine or have locations to a license file or have the license key in the pom.xml that should/can not be public. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (SPARK-33359) foreachBatch sink outputs wrong metrics
John Wesley created SPARK-33359: --- Summary: foreachBatch sink outputs wrong metrics Key: SPARK-33359 URL: https://issues.apache.org/jira/browse/SPARK-33359 Project: Spark Issue Type: Bug Components: Structured Streaming Affects Versions: 3.0.0 Environment: Spark on Kubernetes cluster with spark-3.0.0 image. The CRD is ScheduledSparkApplication Reporter: John Wesley I created 2 similar jobs, 1) First job reading from kafka and writing to console sink in append mode 2) Second job reading from kafka and writing to foreachBatch sink (which then writes in parquet format to S3). The metrics in the log for console shows correct values for numInputRows and numOutputRows whereas they are wrong for foreachBatch. With foreachBatch: numInputRows is +1 more than what is actually present numOutputRows is always -1. ///Console sink //20/11/05 13:36:21 INFO MicroBatchExecution: Streaming query made progress: { "id" : "775aa543-58bf-4cf7-b274-390da640b6ae", "runId" : "e5eac4ca-0b29-4ed4-be35-b70bd20906d5", "name" : null, "timestamp" : "2020-11-05T13:36:08.921Z", "batchId" : 0, "numInputRows" : 10, "processedRowsPerSecond" : 0.7759757895553658, "durationMs" : { "addBatch" : 7735, "getBatch" : 152, "latestOffset" : 2037, "queryPlanning" : 1010, "triggerExecution" : 12886, "walCommit" : 938 }, "stateOperators" : [ ], "sources" : [ { "description" : "KafkaV2[Subscribe[testedr7]]", "startOffset" : null, "endOffset" : { "testedr7" : { "0" : 10 } }, "numInputRows" : 10, "processedRowsPerSecond" : 0.7759757895553658 } ], "sink" : { "description" : "org.apache.spark.sql.execution.streaming.ConsoleTable$@38c3a814", "numOutputRows" : 10 } } ///ForEachBatch Sink //20/11/05 13:43:38 INFO MicroBatchExecution: Streaming query made progress: { "id" : "789f9a00-2f2a-4f75-b643-fea201088b4a", "runId" : "b5e695c5-3a2e-4ad2-9dbf-11b69f368f61", "name" : null, "timestamp" : "2020-11-05T13:43:15.421Z", "batchId" : 0, "numInputRows" : 11, "processedRowsPerSecond" : 0.4833252779120348, "durationMs" : { "addBatch" : 17689, "getBatch" : 135, "latestOffset" : 2121, "queryPlanning" : 880, "triggerExecution" : 22758, "walCommit" : 876 }, "stateOperators" : [ ], "sources" : [ { "description" : "KafkaV2[Subscribe[testedr7]]", "startOffset" : null, "endOffset" : { "testedr7" : { "0" : 10 } }, "numInputRows" : 11, "processedRowsPerSecond" : 0.4833252779120348 } ], "sink" : { "description" : "ForeachBatchSink", "numOutputRows" : -1 } } -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-33361) Dataset.observe() functionality does not work with structured streaming
John Wesley created SPARK-33361: --- Summary: Dataset.observe() functionality does not work with structured streaming Key: SPARK-33361 URL: https://issues.apache.org/jira/browse/SPARK-33361 Project: Spark Issue Type: Bug Components: Structured Streaming Affects Versions: 3.0.0 Environment: Spark on k8s, version 3.0.0 Reporter: John Wesley The dataset observe() functionality does not work as expected with spark in cluster mode. Related discussion here: [http://apache-spark-user-list.1001560.n3.nabble.com/Best-way-to-emit-custom-metrics-to-Prometheus-in-spark-structured-streaming-td38826.html] Using lit() as the aggregation column goes through well. However sum, count etc returns 0 all the time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33359) foreachBatch sink outputs wrong metrics
[ https://issues.apache.org/jira/browse/SPARK-33359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227064#comment-17227064 ] John Wesley commented on SPARK-33359: - Thanks for the clarification. How about numInputRows? > foreachBatch sink outputs wrong metrics > --- > > Key: SPARK-33359 > URL: https://issues.apache.org/jira/browse/SPARK-33359 > Project: Spark > Issue Type: Bug > Components: Structured Streaming >Affects Versions: 3.0.0 > Environment: Spark on Kubernetes cluster with spark-3.0.0 image. The > CRD is ScheduledSparkApplication >Reporter: John Wesley >Priority: Minor > > I created 2 similar jobs, > 1) First job reading from kafka and writing to console sink in append mode > 2) Second job reading from kafka and writing to foreachBatch sink (which then > writes in parquet format to S3). > The metrics in the log for console shows correct values for numInputRows and > numOutputRows whereas they are wrong for foreachBatch. > With foreachBatch: > numInputRows is +1 more than what is actually present > numOutputRows is always -1. > ///Console sink > //20/11/05 13:36:21 INFO > MicroBatchExecution: Streaming query made progress: { > "id" : "775aa543-58bf-4cf7-b274-390da640b6ae", > "runId" : "e5eac4ca-0b29-4ed4-be35-b70bd20906d5", > "name" : null, > "timestamp" : "2020-11-05T13:36:08.921Z", > "batchId" : 0, > "numInputRows" : 10, > "processedRowsPerSecond" : 0.7759757895553658, > "durationMs" : { > "addBatch" : 7735, > "getBatch" : 152, > "latestOffset" : 2037, > "queryPlanning" : 1010, > "triggerExecution" : 12886, > "walCommit" : 938 > }, > "stateOperators" : [ ], > "sources" : [ { > "description" : "KafkaV2[Subscribe[testedr7]]", > "startOffset" : null, > "endOffset" : { > "testedr7" : { > "0" : 10 > } > }, > "numInputRows" : 10, > "processedRowsPerSecond" : 0.7759757895553658 > } ], > "sink" : { > "description" : > "org.apache.spark.sql.execution.streaming.ConsoleTable$@38c3a814", > "numOutputRows" : 10 > } > } > ///ForEachBatch Sink > //20/11/05 13:43:38 INFO > MicroBatchExecution: Streaming query made progress: { > "id" : "789f9a00-2f2a-4f75-b643-fea201088b4a", > "runId" : "b5e695c5-3a2e-4ad2-9dbf-11b69f368f61", > "name" : null, > "timestamp" : "2020-11-05T13:43:15.421Z", > "batchId" : 0, > "numInputRows" : 11, > "processedRowsPerSecond" : 0.4833252779120348, > "durationMs" : { > "addBatch" : 17689, > "getBatch" : 135, > "latestOffset" : 2121, > "queryPlanning" : 880, > "triggerExecution" : 22758, > "walCommit" : 876 > }, > "stateOperators" : [ ], > "sources" : [ { > "description" : "KafkaV2[Subscribe[testedr7]]", > "startOffset" : null, > "endOffset" : { > "testedr7" : { > "0" : 10 > } > }, > "numInputRows" : 11, > "processedRowsPerSecond" : 0.4833252779120348 > } ], > "sink" : { > "description" : "ForeachBatchSink", > "numOutputRows" : -1 > } > } -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-16909) Streaming for postgreSQL JDBC driver
prince john wesley created SPARK-16909: -- Summary: Streaming for postgreSQL JDBC driver Key: SPARK-16909 URL: https://issues.apache.org/jira/browse/SPARK-16909 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 2.0.0 Reporter: prince john wesley postgreSQL JDBC driver sets 0 as default record fetch size, which means, it caches all rows irrespective of the row count. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org