[jira] [Created] (SPARK-46279) Support write partition values to data files
fred liu created SPARK-46279: Summary: Support write partition values to data files Key: SPARK-46279 URL: https://issues.apache.org/jira/browse/SPARK-46279 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 4.0.0 Reporter: fred liu Support write partition values to data files would give the flexibility to allow parquet files to be read correctly without relying on engine to read partition values from the path, and enables cases where individual parquet files can be copied and shared -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (AIRFLOW-4141) Should `run_me_first` dump 3 outlets in `Lineage` document?
[
https://issues.apache.org/jira/browse/AIRFLOW-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fred Liu updated AIRFLOW-4141:
--
Description:
In the latest version(1.10.2) document, [Lineage
chapter|https://airflow.apache.org/lineage.html], the paragraph after the
second {color:#59afe1}Note{color}(which has blue banner) contains this line:
... run_me_first is a BashOperator that {color:#ff}takes 3 inlets{color}:
CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#ff}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
was:
In the latest version(1.10.2) document, [Lineage
chapter|https://airflow.apache.org/lineage.html], the paragraph after the
second Note(which has blue banner) contains this line: ... run_me_first is a
BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#ff}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
> Should `run_me_first` dump 3 outlets in `Lineage` document?
> ---
>
> Key: AIRFLOW-4141
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4141
> Project: Apache Airflow
> Issue Type: Bug
> Components: docs, Documentation
>Affects Versions: 1.10.2
>Reporter: Fred Liu
>Priority: Trivial
> Labels: easyfix
>
> In the latest version(1.10.2) document, [Lineage
> chapter|https://airflow.apache.org/lineage.html], the paragraph after the
> second {color:#59afe1}Note{color}(which has blue banner) contains this line:
> ... run_me_first is a BashOperator that {color:#ff}takes 3 inlets{color}:
> CAT1, CAT2, CAT3 ...
> We can figure out CATx are defined as outlets instead inlets from example
> code above.
> I think "run_me_first is a BashOperator that {color:#ff}dumps 3
> outlets{color}: CAT1, CAT2, CAT3" is better?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4141) Should `run_me_first` dump 3 outlets in `Lineage` document?
[
https://issues.apache.org/jira/browse/AIRFLOW-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fred Liu updated AIRFLOW-4141:
--
Description:
In the latest version(1.10.2) document, [Lineage
chapter|[https://airflow.apache.org/lineage.html]], the paragraph after the
second Note(which has blue banner) contains this line: ... run_me_first is a
BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#ff}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
was:
In the latest version(1.10.2) document, Lineage chapter, the paragraph after
the second Note(which has blue banner) contains this line: ... run_me_first is
a BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#ff}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
> Should `run_me_first` dump 3 outlets in `Lineage` document?
> ---
>
> Key: AIRFLOW-4141
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4141
> Project: Apache Airflow
> Issue Type: Bug
> Components: docs, Documentation
>Affects Versions: 1.10.2
>Reporter: Fred Liu
>Priority: Trivial
> Labels: easyfix
>
> In the latest version(1.10.2) document, [Lineage
> chapter|[https://airflow.apache.org/lineage.html]], the paragraph after the
> second Note(which has blue banner) contains this line: ... run_me_first is a
> BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
> We can figure out CATx are defined as outlets instead inlets from example
> code above.
> I think "run_me_first is a BashOperator that {color:#ff}dumps 3
> outlets{color}: CAT1, CAT2, CAT3" is better?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4141) Should `run_me_first` dump 3 outlets in `Lineage` document?
[
https://issues.apache.org/jira/browse/AIRFLOW-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fred Liu updated AIRFLOW-4141:
--
Description:
In the latest version(1.10.2) document, [Lineage
chapter|https://airflow.apache.org/lineage.html], the paragraph after the
second Note(which has blue banner) contains this line: ... run_me_first is a
BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#ff}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
was:
In the latest version(1.10.2) document, [Lineage
chapter|[https://airflow.apache.org/lineage.html]], the paragraph after the
second Note(which has blue banner) contains this line: ... run_me_first is a
BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#ff}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
> Should `run_me_first` dump 3 outlets in `Lineage` document?
> ---
>
> Key: AIRFLOW-4141
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4141
> Project: Apache Airflow
> Issue Type: Bug
> Components: docs, Documentation
>Affects Versions: 1.10.2
>Reporter: Fred Liu
>Priority: Trivial
> Labels: easyfix
>
> In the latest version(1.10.2) document, [Lineage
> chapter|https://airflow.apache.org/lineage.html], the paragraph after the
> second Note(which has blue banner) contains this line: ... run_me_first is a
> BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
> We can figure out CATx are defined as outlets instead inlets from example
> code above.
> I think "run_me_first is a BashOperator that {color:#ff}dumps 3
> outlets{color}: CAT1, CAT2, CAT3" is better?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4141) Should `run_me_first` dump 3 outlets in `Lineage` document?
[
https://issues.apache.org/jira/browse/AIRFLOW-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fred Liu updated AIRFLOW-4141:
--
Description:
In the latest version(1.10.2) document, Lineage chapter, the paragraph after
the second Note(which has blue banner) contains this line: ... run_me_first is
a BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#ff}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
was:
In latest version document, Lineage chapter, the paragraph after the second
Note(which has blue banner) contains this line: ... run_me_first is a
BashOperator that {color:#FF}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#FF}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
> Should `run_me_first` dump 3 outlets in `Lineage` document?
> ---
>
> Key: AIRFLOW-4141
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4141
> Project: Apache Airflow
> Issue Type: Bug
> Components: docs, Documentation
>Affects Versions: 1.10.2
>Reporter: Fred Liu
>Priority: Trivial
> Labels: easyfix
>
> In the latest version(1.10.2) document, Lineage chapter, the paragraph after
> the second Note(which has blue banner) contains this line: ... run_me_first
> is a BashOperator that {color:#ff}takes 3 inlets{color}: CAT1, CAT2, CAT3
> ...
> We can figure out CATx are defined as outlets instead inlets from example
> code above.
> I think "run_me_first is a BashOperator that {color:#ff}dumps 3
> outlets{color}: CAT1, CAT2, CAT3" is better?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-4141) Should `run_me_first` dump 3 outlets in `Lineage` document?
Fred Liu created AIRFLOW-4141:
-
Summary: Should `run_me_first` dump 3 outlets in `Lineage`
document?
Key: AIRFLOW-4141
URL: https://issues.apache.org/jira/browse/AIRFLOW-4141
Project: Apache Airflow
Issue Type: Bug
Components: docs, Documentation
Affects Versions: 1.10.2
Reporter: Fred Liu
In latest version document, Lineage chapter, the paragraph after the second
Note(which has blue banner) contains this line: ... run_me_first is a
BashOperator that {color:#FF}takes 3 inlets{color}: CAT1, CAT2, CAT3 ...
We can figure out CATx are defined as outlets instead inlets from example code
above.
I think "run_me_first is a BashOperator that {color:#FF}dumps 3
outlets{color}: CAT1, CAT2, CAT3" is better?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4845) Add JanusGraph put processor
[ https://issues.apache.org/jira/browse/NIFI-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358015#comment-16358015 ] Fred Liu commented on NIFI-4845: Hi [~mcgilman], Can you assign this task to me? Is this new processor necessary? > Add JanusGraph put processor > > > Key: NIFI-4845 > URL: https://issues.apache.org/jira/browse/NIFI-4845 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Fred Liu >Priority: Major > > Create processor for Reading records from an incoming FlowFile using the > provided Record Reader, and writting those records to JanusGraph. And using a > JanusGraphControllerService is good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4845) Add JanusGraph put processor
[ https://issues.apache.org/jira/browse/NIFI-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353509#comment-16353509 ] Fred Liu commented on NIFI-4845: Hi Admin, Please assign this task to me so that I can begin working on the implementation. > Add JanusGraph put processor > > > Key: NIFI-4845 > URL: https://issues.apache.org/jira/browse/NIFI-4845 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Fred Liu >Priority: Major > > Create processor for Reading records from an incoming FlowFile using the > provided Record Reader, and writting those records to JanusGraph. And using a > JanusGraphControllerService is good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4845) Add JanusGraph put processor
Fred Liu created NIFI-4845: -- Summary: Add JanusGraph put processor Key: NIFI-4845 URL: https://issues.apache.org/jira/browse/NIFI-4845 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Fred Liu Create processor for Reading records from an incoming FlowFile using the provided Record Reader, and writting those records to JanusGraph. And using a JanusGraphControllerService is good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4494) Add a FetchOracleRow processor
Fred Liu created NIFI-4494: -- Summary: Add a FetchOracleRow processor Key: NIFI-4494 URL: https://issues.apache.org/jira/browse/NIFI-4494 Project: Apache NiFi Issue Type: New Feature Components: Extensions Environment: oracle Reporter: Fred Liu We encounter a lot of demand, poor data quality, no primary key, no time stamp, and even a lot of duplicate data. But the customer requires a high performance and accuracy. Using GenerateTableFetch or QueryDatabaseTable, we can not meet the functional and performance requirements. So we want to add a new processor, it is specifically for the oracle database, able to ingest very poor quality data and have better performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1031) 'scheduled__' may replace with DagRun.ID_PREFIX in DagRun.run_id
Fred Liu created AIRFLOW-1031: - Summary: 'scheduled__' may replace with DagRun.ID_PREFIX in DagRun.run_id Key: AIRFLOW-1031 URL: https://issues.apache.org/jira/browse/AIRFLOW-1031 Project: Apache Airflow Issue Type: Improvement Components: DagRun Affects Versions: Airflow 1.8 Reporter: Fred Liu Assignee: Fred Liu Priority: Trivial Fix For: Airflow 1.8 https://github.com/apache/incubator-airflow/blob/v1-8-stable/airflow/jobs.py#L840 This 'scheduled__' may replace with DagRun.ID_PREFIX I searched 'scheduled__', it seems don't appear other code. Can I send a PR to fix this? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949890#comment-14949890 ] Fred Liu commented on HBASE-13858: -- The patch added one line to flush the output and no unit test needed. > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fred Liu updated HBASE-13858: - Attachment: HBASE-13858.patch add flush after outputing stack header > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fred Liu updated HBASE-13858: - Status: Patch Available (was: Open) add flush after outputing stack header > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
