[jira] [Resolved] (APEXCORE-792) LoggerUtil should allow to get LogFileInformation for a specified logger
[ https://issues.apache.org/jira/browse/APEXCORE-792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale resolved APEXCORE-792. -- Resolution: Fixed Fix Version/s: 3.7.0 > LoggerUtil should allow to get LogFileInformation for a specified logger > - > > Key: APEXCORE-792 > URL: https://issues.apache.org/jira/browse/APEXCORE-792 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > Fix For: 3.7.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (APEXCORE-792) LoggerUtil should allow to get LogFileInformation for a specified logger
[ https://issues.apache.org/jira/browse/APEXCORE-792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233667#comment-16233667 ] Priyanka Gugale commented on APEXCORE-792: -- Can you please specify use case for this change? Also I am not aware of scenarios where multiple loggers are used at same time. Can you share any example? > LoggerUtil should allow to get LogFileInformation for a specified logger > - > > Key: APEXCORE-792 > URL: https://issues.apache.org/jira/browse/APEXCORE-792 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (APEXMALHAR-2535) Timeouts in AbstractEnricher specified as int which limits duration of time which could be specified.
[ https://issues.apache.org/jira/browse/APEXMALHAR-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale resolved APEXMALHAR-2535. - Resolution: Fixed Fix Version/s: 3.8.0 > Timeouts in AbstractEnricher specified as int which limits duration of time > which could be specified. > - > > Key: APEXMALHAR-2535 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2535 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Tushar Gosavi >Priority: Minor > Fix For: 3.8.0 > > > cacheExpirationInterval and cacheCleanupInterval timeout properties are using > int as datatype which limits the amount of time duration could be specified > using them. I am > going to change they to long also provide a way for operator to specify > expiryType on cacheStore -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (APEXMALHAR-2535) Timeouts in AbstractEnricher specified as int which limits duration of time which could be specified.
[ https://issues.apache.org/jira/browse/APEXMALHAR-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale reassigned APEXMALHAR-2535: --- Assignee: Tushar Gosavi > Timeouts in AbstractEnricher specified as int which limits duration of time > which could be specified. > - > > Key: APEXMALHAR-2535 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2535 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Tushar Gosavi >Assignee: Tushar Gosavi >Priority: Minor > Fix For: 3.8.0 > > > cacheExpirationInterval and cacheCleanupInterval timeout properties are using > int as datatype which limits the amount of time duration could be specified > using them. I am > going to change they to long also provide a way for operator to specify > expiryType on cacheStore -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (APEXCORE-602) Provide a "group-id" in the event object so that events are grouped together by a "root cause".
[ https://issues.apache.org/jira/browse/APEXCORE-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085875#comment-16085875 ] Priyanka Gugale commented on APEXCORE-602: -- Testing covered: 1. Wrote unit tests 2. Simulated operator failure recursively where before redeploy on a operator is finished it fails again. This creates multiple deploy request in parallel for a operator but different containers. (Forced to throw exception on certain conditions). 3. Manually killed a operator and predecessor operator within very short duration to make same operators redeploy in multiple requests. 4. Random killing of operator/containers for testing. > Provide a "group-id" in the event object so that events are grouped together > by a "root cause". > --- > > Key: APEXCORE-602 > URL: https://issues.apache.org/jira/browse/APEXCORE-602 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sanjay M Pujare >Assignee: Priyanka Gugale > Fix For: 3.7.0 > > > Provide a "group-id" in the event object so that events are grouped together > by a "root cause". An example is a bunch of container restarts are related to > a single failure in the application but the current sequence of Stram events > doesn't make it obvious. The consumer of events is able to better > read/analyze the events because of the group-id and focus on the root-cause. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (APEXCORE-756) Fix ConcurrentModificationException in GroupingManager
[ https://issues.apache.org/jira/browse/APEXCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale updated APEXCORE-756: - Summary: Fix ConcurrentModificationException in GroupingManager (was: Fix test failure in GroupingManager) > Fix ConcurrentModificationException in GroupingManager > -- > > Key: APEXCORE-756 > URL: https://issues.apache.org/jira/browse/APEXCORE-756 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > At times the StreamingContainerManager test fails as: > com.datatorrent.stram.StreamingContainerManagerTest > testShutdownOperatorRecovery(com.datatorrent.stram.StreamingContainerManagerTest) > Time elapsed: 0.046 sec <<< ERROR! > java.util.ConcurrentModificationException > at > com.datatorrent.stram.StreamingContainerManagerTest.testShutdownOperatorRecovery(StreamingContainerManagerTest.java:464) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (APEXCORE-756) Fix test failure in GroupingManager
[ https://issues.apache.org/jira/browse/APEXCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale updated APEXCORE-756: - Summary: Fix test failure in GroupingManager (was: Fix test failure in GropingManager) > Fix test failure in GroupingManager > --- > > Key: APEXCORE-756 > URL: https://issues.apache.org/jira/browse/APEXCORE-756 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > At times the StreamingContainerManager test fails as: > com.datatorrent.stram.StreamingContainerManagerTest > testShutdownOperatorRecovery(com.datatorrent.stram.StreamingContainerManagerTest) > Time elapsed: 0.046 sec <<< ERROR! > java.util.ConcurrentModificationException > at > com.datatorrent.stram.StreamingContainerManagerTest.testShutdownOperatorRecovery(StreamingContainerManagerTest.java:464) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (APEXCORE-756) Fix test failure in GropingManager
Priyanka Gugale created APEXCORE-756: Summary: Fix test failure in GropingManager Key: APEXCORE-756 URL: https://issues.apache.org/jira/browse/APEXCORE-756 Project: Apache Apex Core Issue Type: Bug Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor At times the StreamingContainerManager test fails as: com.datatorrent.stram.StreamingContainerManagerTest testShutdownOperatorRecovery(com.datatorrent.stram.StreamingContainerManagerTest) Time elapsed: 0.046 sec <<< ERROR! java.util.ConcurrentModificationException at com.datatorrent.stram.StreamingContainerManagerTest.testShutdownOperatorRecovery(StreamingContainerManagerTest.java:464) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (APEXMALHAR-2447) No indication from AbstractFileInputOperator when directory is empty
[ https://issues.apache.org/jira/browse/APEXMALHAR-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale resolved APEXMALHAR-2447. - Resolution: Fixed Fix Version/s: 3.8.0 > No indication from AbstractFileInputOperator when directory is empty > > > Key: APEXMALHAR-2447 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2447 > Project: Apache Apex Malhar > Issue Type: Improvement >Reporter: Velineni Lakshmi Prasanna >Assignee: Velineni Lakshmi Prasanna >Priority: Minor > Fix For: 3.8.0 > > > AbstractFileInputOperator scans a directory for files and outputs the file > contents. However, when the directory is empty nothing is sent. A downstream > operator may want some basic information that the input directory is scanned, > so for example, it can create an output folder. Add a notification to the > operator when the directory is scanned for the first time. An implementation, > for example, can use this notification to send a control tuple with the > directory path notifying the downstream that the directory has been visited. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXMALHAR-2487) Malhar should support outputting data in Snappy compression
[ https://issues.apache.org/jira/browse/APEXMALHAR-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale updated APEXMALHAR-2487: Fix Version/s: 3.8.0 > Malhar should support outputting data in Snappy compression > --- > > Key: APEXMALHAR-2487 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2487 > Project: Apache Apex Malhar > Issue Type: Improvement >Reporter: Ilya Ganelin >Assignee: Ilya Ganelin > Fix For: 3.8.0 > > > At present, the default file output operator (AbstractFileOutputOperator) > supports compression by setting the FilterStreamProvider. However, Malhar > presently only includes two FilterStreamProvider - one to Cipher data, and > one for Gzip. > Snappy offers substantially improved performance over Gzip in terms of > compression and decompression speed at the expense of compression ratio. In > certain applications this is useful. Thus, it would be helpful to add a > Snappy FilterStreamProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (APEXMALHAR-2487) Malhar should support outputting data in Snappy compression
[ https://issues.apache.org/jira/browse/APEXMALHAR-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale resolved APEXMALHAR-2487. - Resolution: Fixed > Malhar should support outputting data in Snappy compression > --- > > Key: APEXMALHAR-2487 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2487 > Project: Apache Apex Malhar > Issue Type: Improvement >Reporter: Ilya Ganelin >Assignee: Ilya Ganelin > Fix For: 3.8.0 > > > At present, the default file output operator (AbstractFileOutputOperator) > supports compression by setting the FilterStreamProvider. However, Malhar > presently only includes two FilterStreamProvider - one to Cipher data, and > one for Gzip. > Snappy offers substantially improved performance over Gzip in terms of > compression and decompression speed at the expense of compression ratio. In > certain applications this is useful. Thus, it would be helpful to add a > Snappy FilterStreamProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (APEXMALHAR-2418) Update the twitter library to 4.0.6
[ https://issues.apache.org/jira/browse/APEXMALHAR-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale resolved APEXMALHAR-2418. - Resolution: Fixed Assignee: (was: Priyanka Gugale) Fix Version/s: 3.7.0 > Update the twitter library to 4.0.6 > > > Key: APEXMALHAR-2418 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2418 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Shubham Agrawal >Priority: Minor > Fix For: 3.7.0 > > > There was a known issue of Build Failure for twitter version 4.0.4. > So for succesfully building and launching the app need to change the version > from 4.0.4 to 4.0.6. > It has been tested. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (APEXMALHAR-2418) Update the twitter library to 4.0.6
[ https://issues.apache.org/jira/browse/APEXMALHAR-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale reassigned APEXMALHAR-2418: --- Assignee: Priyanka Gugale > Update the twitter library to 4.0.6 > > > Key: APEXMALHAR-2418 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2418 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Shubham Agrawal >Assignee: Priyanka Gugale >Priority: Minor > > There was a known issue of Build Failure for twitter version 4.0.4. > So for succesfully building and launching the app need to change the version > from 4.0.4 to 4.0.6. > It has been tested. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (APEXCORE-602) Provide a "group-id" in the event object so that events are grouped together by a "root cause".
[ https://issues.apache.org/jira/browse/APEXCORE-602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale reassigned APEXCORE-602: Assignee: Priyanka Gugale > Provide a "group-id" in the event object so that events are grouped together > by a "root cause". > --- > > Key: APEXCORE-602 > URL: https://issues.apache.org/jira/browse/APEXCORE-602 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sanjay M Pujare >Assignee: Priyanka Gugale > > Provide a "group-id" in the event object so that events are grouped together > by a "root cause". An example is a bunch of container restarts are related to > a single failure in the application but the current sequence of Stram events > doesn't make it obvious. The consumer of events is able to better > read/analyze the events because of the group-id and focus on the root-cause. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2367) CassandraTransactionalStore should handle erroneous records
Priyanka Gugale created APEXMALHAR-2367: --- Summary: CassandraTransactionalStore should handle erroneous records Key: APEXMALHAR-2367 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2367 Project: Apache Apex Malhar Issue Type: Improvement Reporter: Priyanka Gugale Priority: Minor If CassandraTransactionalStore batch command execution fails, there should be a way to configure retry attempt and after that the statement / batch should be dropped or emitted to error port without blocking application execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (APEXMALHAR-2330) JdbcPOJOPollInputOperator fails with NullPointerException when PostgreSQL driver
[ https://issues.apache.org/jira/browse/APEXMALHAR-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale resolved APEXMALHAR-2330. - Resolution: Fixed Fix Version/s: 3.7.0 > JdbcPOJOPollInputOperator fails with NullPointerException when PostgreSQL > driver > > > Key: APEXMALHAR-2330 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2330 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Deepak Narkhede >Assignee: Deepak Narkhede > Fix For: 3.7.0 > > > Here is the description: > Problem: > JdbcPOJOPollInputOperator fails with NullPointerException when PostgreSQL > driver. > Problem Description: > 1) When JdbcPOJOPollInputOperator tries to populateColumnDataTypes, column > names retrieved from resultmetadata from database ( this case : Postgres) are > all lowercase. > 2) Whereas columnDatatypes specified in fieldinfos might be in same case. > 3) Internally hashmap ( nameToType) is used which mismatches if column name > and fieldinfo are not in same case. Hence columnDataTypes is empty which > causes null exception in activate call. > Proposed Solution: > Using similar case for hashmap and column names irrespective of any database > used for JdbcPOJOPollInputOperator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2356) Amend javadoc syntax in cassandra upsert operators
Priyanka Gugale created APEXMALHAR-2356: --- Summary: Amend javadoc syntax in cassandra upsert operators Key: APEXMALHAR-2356 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2356 Project: Apache Apex Malhar Issue Type: Bug Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (APEXCORE-578) Amend javadoc syntax in AbstractUpsertOutputOperator
[ https://issues.apache.org/jira/browse/APEXCORE-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale closed APEXCORE-578. Resolution: Invalid Opened in error. > Amend javadoc syntax in AbstractUpsertOutputOperator > > > Key: APEXCORE-578 > URL: https://issues.apache.org/jira/browse/APEXCORE-578 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (APEXMALHAR-2325) Same block id is emitting from FSInputModule
[
https://issues.apache.org/jira/browse/APEXMALHAR-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanka Gugale resolved APEXMALHAR-2325.
-
Resolution: Fixed
Fix Version/s: 3.6.0
> Same block id is emitting from FSInputModule
>
>
> Key: APEXMALHAR-2325
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2325
> Project: Apache Apex Malhar
> Issue Type: Bug
>Reporter: Chaitanya
>Assignee: Chaitanya
>Priority: Minor
> Fix For: 3.6.0
>
>
> Observation: Mismatch the block size between the filesplitter and block
> reader in FSInput Module.
> Default block size in block reader = conf.getLong("fs.local.block.size" )
> i.e, Local file system block size.
> Default block size in filesplitter = fs.getDefaultBlockSize(File Path)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (APEXCORE-561) Container info needs to be persisted even after application has been killed
[ https://issues.apache.org/jira/browse/APEXCORE-561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale reassigned APEXCORE-561: Assignee: Priyanka Gugale > Container info needs to be persisted even after application has been killed > --- > > Key: APEXCORE-561 > URL: https://issues.apache.org/jira/browse/APEXCORE-561 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sanjay M Pujare >Assignee: Priyanka Gugale > > This is required so that info can be displayed (by Apex cli for example) even > for killed apps. > At least one piece of info missing, which is the finishedTime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (APEXCORE-564) We should be able to remove loggers level settings
[ https://issues.apache.org/jira/browse/APEXCORE-564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale closed APEXCORE-564. Resolution: Won't Fix > We should be able to remove loggers level settings > -- > > Key: APEXCORE-564 > URL: https://issues.apache.org/jira/browse/APEXCORE-564 > Project: Apache Apex Core > Issue Type: New Feature >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > Original Estimate: 48h > Remaining Estimate: 48h > > StramWebServices has api to setLoggersLevel, but there is not way to remove > the already set loggers level. There should be an api to remove loggers level > which was set earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXCORE-564) We should be able to remove loggers level settings
[ https://issues.apache.org/jira/browse/APEXCORE-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605201#comment-15605201 ] Priyanka Gugale commented on APEXCORE-564: -- okay closing the issue in that case. > We should be able to remove loggers level settings > -- > > Key: APEXCORE-564 > URL: https://issues.apache.org/jira/browse/APEXCORE-564 > Project: Apache Apex Core > Issue Type: New Feature >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > Original Estimate: 48h > Remaining Estimate: 48h > > StramWebServices has api to setLoggersLevel, but there is not way to remove > the already set loggers level. There should be an api to remove loggers level > which was set earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXCORE-564) We should be able to remove loggers level settings
[ https://issues.apache.org/jira/browse/APEXCORE-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601132#comment-15601132 ] Priyanka Gugale commented on APEXCORE-564: -- Our initial though was, we will set it to NULL so that either parent or root level logger will be picked up. As you explained patters are cumulative so taking example of com.datatorrent.engine.*, say following levels were set earlier com.datatorrent.engine.*=INFO com.datatorrent.engine.*=DEBUG Now to remove it, as LoggerUtil doesn't have a way to remove these patterns, we though of setting ti to null so now entries look like com.datatorrent.engine.*=INFO com.datatorrent.engine.*=DEBUG com.datatorrent.engine.*=null After doing the logger level for com.datatorrent.engine.* will either be picked from parent pattern say com.datatorrent.* or from root logger. That was our understanding. Also after this if someone wishes to set some new level on logger that is also feasible. Please let us know if this approach looks good? > We should be able to remove loggers level settings > -- > > Key: APEXCORE-564 > URL: https://issues.apache.org/jira/browse/APEXCORE-564 > Project: Apache Apex Core > Issue Type: New Feature >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > Original Estimate: 48h > Remaining Estimate: 48h > > StramWebServices has api to setLoggersLevel, but there is not way to remove > the already set loggers level. There should be an api to remove loggers level > which was set earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXCORE-564) We should be able to remove loggers level settings
Priyanka Gugale created APEXCORE-564: Summary: We should be able to remove loggers level settings Key: APEXCORE-564 URL: https://issues.apache.org/jira/browse/APEXCORE-564 Project: Apache Apex Core Issue Type: New Feature Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor StramWebServices has api to setLoggersLevel, but there is not way to remove the already set loggers level. There should be an api to remove loggers level which was set earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXCORE-562) RecordingsAgent: returns records for offset beyond number of tuples
[ https://issues.apache.org/jira/browse/APEXCORE-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588709#comment-15588709 ] Priyanka Gugale commented on APEXCORE-562: -- [~thw] please add me as contributor in the JIRA project so that I can assign this ticket to myself. > RecordingsAgent: returns records for offset beyond number of tuples > --- > > Key: APEXCORE-562 > URL: https://issues.apache.org/jira/browse/APEXCORE-562 > Project: Apache Apex Core > Issue Type: Bug >Affects Versions: 3.4.0 >Reporter: Priyanka Gugale >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > When we call "getTuplesInfoByOffset" function with offset value greater than > total number of tuples in recording it returns the records. > Expected behavior: No records should be returned as offset is greater than > available number of records. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (APEXCORE-562) RecordingsAgent: returns records for offset beyond number of tuples
[ https://issues.apache.org/jira/browse/APEXCORE-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale updated APEXCORE-562: - Affects Version/s: 3.4.0 > RecordingsAgent: returns records for offset beyond number of tuples > --- > > Key: APEXCORE-562 > URL: https://issues.apache.org/jira/browse/APEXCORE-562 > Project: Apache Apex Core > Issue Type: Bug >Affects Versions: 3.4.0 >Reporter: Priyanka Gugale >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > When we call "getTuplesInfoByOffset" function with offset value greater than > total number of tuples in recording it returns the records. > Expected behavior: No records should be returned as offset is greater than > available number of records. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXCORE-562) RecordingsAgent: returns records for offset beyond number of tuples
[ https://issues.apache.org/jira/browse/APEXCORE-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588531#comment-15588531 ] Priyanka Gugale commented on APEXCORE-562: -- getTuplesInfo function incorrectly updates current offset even if it doesn't read any records. > RecordingsAgent: returns records for offset beyond number of tuples > --- > > Key: APEXCORE-562 > URL: https://issues.apache.org/jira/browse/APEXCORE-562 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Priyanka Gugale >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > When we call "getTuplesInfoByOffset" function with offset value greater than > total number of tuples in recording it returns the records. > Expected behavior: No records should be returned as offset is greater than > available number of records. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2272) sequentialFileRead property on FSInputModule not functioning as expected
[ https://issues.apache.org/jira/browse/APEXMALHAR-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584507#comment-15584507 ] Priyanka Gugale commented on APEXMALHAR-2272: - done. > sequentialFileRead property on FSInputModule not functioning as expected > > > Key: APEXMALHAR-2272 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2272 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Yogi Devendra >Assignee: Yogi Devendra >Priority: Minor > Fix For: 3.6.0 > > > When there is single large file in the input directory, and we have multiple > partitions for BlockReader and sequencialFileRead is set to true. > Only one BlockReader instance should be active; other BlockReader instances > should remain idle. > This is because sequencialFileRead makes sure that blocks of the file are > read serially by same BlockReader instance. > Observed behavior is all BlockReader instances are reading data which means > sequencialFileRead property is not functioning as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (APEXMALHAR-2255) Use latest couchbase client library
[ https://issues.apache.org/jira/browse/APEXMALHAR-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale updated APEXMALHAR-2255: Summary: Use latest couchbase client library (was: Use latest java sdk for couchbase) > Use latest couchbase client library > --- > > Key: APEXMALHAR-2255 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2255 > Project: Apache Apex Malhar > Issue Type: Test >Reporter: Priyanka Gugale > > Right now the Couchbase connector in Malhar uses couchbase-client library > which is outdated. We should instead use java-client library from couchbase > to connect to couchbase server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2255) Use latest java sdk for couchbase
[ https://issues.apache.org/jira/browse/APEXMALHAR-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512385#comment-15512385 ] Priyanka Gugale commented on APEXMALHAR-2255: - The request is to upgrade *couchbase java sdk* i.e. couchbase client library. I will update the title. Here is the compatibility matrix: http://developer.couchbase.com/documentation/server/current/sdk/java/compatibility-versions-features.html I would suggest we should pick the latest at time from: https://mvnrepository.com/artifact/com.couchbase.client/java-client (by checking compatibility from above link) > Use latest java sdk for couchbase > - > > Key: APEXMALHAR-2255 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2255 > Project: Apache Apex Malhar > Issue Type: Test >Reporter: Priyanka Gugale > > Right now the Couchbase connector in Malhar uses couchbase-client library > which is outdated. We should instead use java-client library from couchbase > to connect to couchbase server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2255) Use latest java sdk for couchbase
Priyanka Gugale created APEXMALHAR-2255: --- Summary: Use latest java sdk for couchbase Key: APEXMALHAR-2255 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2255 Project: Apache Apex Malhar Issue Type: Test Reporter: Priyanka Gugale Right now the Couchbase connector in Malhar uses couchbase-client library which is outdated. We should instead use java-client library from couchbase to connect to couchbase server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (APEXMALHAR-2232) Add documentation for csv parser
[ https://issues.apache.org/jira/browse/APEXMALHAR-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale updated APEXMALHAR-2232: Priority: Minor (was: Major) > Add documentation for csv parser > > > Key: APEXMALHAR-2232 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2232 > Project: Apache Apex Malhar > Issue Type: Documentation >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2232) Add documentation for csv parser
Priyanka Gugale created APEXMALHAR-2232: --- Summary: Add documentation for csv parser Key: APEXMALHAR-2232 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2232 Project: Apache Apex Malhar Issue Type: Documentation Reporter: Priyanka Gugale Assignee: Priyanka Gugale -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2184) Add documentation for FileSystem Input Operator
Priyanka Gugale created APEXMALHAR-2184: --- Summary: Add documentation for FileSystem Input Operator Key: APEXMALHAR-2184 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2184 Project: Apache Apex Malhar Issue Type: Documentation Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2179) Add documentation for JDBCPollInputOperator
Priyanka Gugale created APEXMALHAR-2179: --- Summary: Add documentation for JDBCPollInputOperator Key: APEXMALHAR-2179 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2179 Project: Apache Apex Malhar Issue Type: Documentation Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues
Priyanka Gugale created APEXMALHAR-2172: --- Summary: Update JDBC poll input operator to fix issues Key: APEXMALHAR-2172 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2172 Project: Apache Apex Malhar Issue Type: Improvement Reporter: Priyanka Gugale Assignee: Priyanka Gugale Update JDBCPollInputOperator to: 1. Fix small bugs 2. Use jooq query dsl library to construct sql queries 3. Make code more readable 4. Use row counts rather than key column values to partition reads -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2163) User query dsl api to construct SQL query
Priyanka Gugale created APEXMALHAR-2163: --- Summary: User query dsl api to construct SQL query Key: APEXMALHAR-2163 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2163 Project: Apache Apex Malhar Issue Type: Improvement Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor JdbcMetaDataUtility class is used to construct the sql queries. The code can supported limited databases and doesn't lots of manipulation to construct query. We should use some good third part dsl api library which will let us write DB agnostic sql queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (APEXMALHAR-2161) Add tests for AbstractThroughputFileInputOperator
[ https://issues.apache.org/jira/browse/APEXMALHAR-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale updated APEXMALHAR-2161: Summary: Add tests for AbstractThroughputFileInputOperator (was: Add tests fir AbstractThroughputFileInputOperator) > Add tests for AbstractThroughputFileInputOperator > - > > Key: APEXMALHAR-2161 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2161 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > Add unit tests for AbstractThroughputFileInputOperator -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2161) Add tests fir AbstractThroughputFileInputOperator
Priyanka Gugale created APEXMALHAR-2161: --- Summary: Add tests fir AbstractThroughputFileInputOperator Key: APEXMALHAR-2161 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2161 Project: Apache Apex Malhar Issue Type: Bug Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor Add unit tests for AbstractThroughputFileInputOperator -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (APEXMALHAR-2121) KafkaInputOperator emitTuple method should be able to emit more than just message
Priyanka Gugale created APEXMALHAR-2121: --- Summary: KafkaInputOperator emitTuple method should be able to emit more than just message Key: APEXMALHAR-2121 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2121 Project: Apache Apex Malhar Issue Type: Improvement Reporter: Priyanka Gugale Assignee: Priyanka Gugale Priority: Minor AbstractKafkaInputOperator in malhar-contrib operator had emitTuple method with parameter as "Message". User can't extend this method to access more kafka parameters other than messages. We should have a method to access to "KafkaMessage" so that user can choose from multiple fields which he/she wants to emit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (APEXMALHAR-2041) Write Cassandra application tests using embedded server
[ https://issues.apache.org/jira/browse/APEXMALHAR-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Gugale reassigned APEXMALHAR-2041: --- Assignee: Priyanka Gugale > Write Cassandra application tests using embedded server > --- > > Key: APEXMALHAR-2041 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2041 > Project: Apache Apex Malhar > Issue Type: Improvement >Reporter: Priyanka Gugale >Assignee: Priyanka Gugale > > We should write cassandra operator and application tests using cassandra > embedded server. One of the available cassandra embedded servers: > https://github.com/jsevellec/cassandra-unit/wiki/How-to-use-it-in-your-code -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2103) scanner issues in FileSplitterInput class
[ https://issues.apache.org/jira/browse/APEXMALHAR-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308863#comment-15308863 ] Priyanka Gugale commented on APEXMALHAR-2103: - Hi, I think the problem is not with wrong keys in general. Key is wrong only in case where input is absolute file path and not the directory path. When input is a file, we don't set it's directoryPath in metadata and that is causing the problem. I would suggest we should fix it by hadling reference time datastructure for file inputs. Here is the code changes for what I am suggesting: https://github.com/apache/incubator-apex-malhar/compare/master...DT-Priyanka:APEXMALHAR-2103-fix-for-file-input > scanner issues in FileSplitterInput class > - > > Key: APEXMALHAR-2103 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2103 > Project: Apache Apex Malhar > Issue Type: Bug >Reporter: Chaitanya >Assignee: Chaitanya > > Issue: FileSplitter continuously emitting filemetadata even though there is > a single file. > Observation: For the same file, While updating and accessing the > referenceTimes map in FIleSplitterInput and TimeBasedScanner, the Keys are > different. Because of this, the oldestTimeModification is always null in > TimeBasedScanner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
