[jira] [Created] (NIFI-12624) ReportingTask: AWSCloudWatchReporterTask

Jorge Machado created NIFI-12624: Summary: ReportingTask: AWSCloudWatchReporterTask Key: NIFI-12624 URL: https://issues.apache.org/jira/browse/NIFI-12624 Project: Apache NiFi Issue Type

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Description: Hello all,    I have a client that has a column named : bfzgtäeil Spark

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Description: Hello all,    I have a client that has a column named : bfzgtäeil Spark

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Priority: Major (was: Minor) > Cannot parse Type from german &quo

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Priority: Minor (was: Critical) > Cannot parse Type from german &quo

[jira] [Created] (SPARK-44108) Cannot parse Type from german "umlaut"

Jorge Machado created SPARK-44108: - Summary: Cannot parse Type from german "umlaut" Key: SPARK-44108 URL: https://issues.apache.org/jira/browse/SPARK-44108 Project: Spark Issue

Unsubscribe

[jira] [Comment Edited] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653941#comment-17653941 ] Jorge Machado edited comment on SPARK-33772 at 1/3/23 12:27 PM: I still

[jira] [Comment Edited] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653941#comment-17653941 ] Jorge Machado edited comment on SPARK-33772 at 1/3/23 12:27 PM: I still

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653941#comment-17653941 ] Jorge Machado commented on SPARK-33772: --- I still have an issue with this. Running sbt test fails

Re: Nifi flowfile monitoring

Activate prometheus exporter and you will have a metric for number of objects in the queue > On 31. Dec 2022, at 07:08, nayan sharma wrote: > > Hi Users, > Is there anyway through which I can monitor or raise alert if any flow file > got stuck in nifi queue. > > For now operation team

Re: Failing to start - keystore properties invalid

Hi James, Can it be that you are trying to start nifi with ssl without authentication ? Looks like that.. > On 27. Dec 2022, at 22:13, James McMahon wrote: > > Hello. I am trying to start a secure instance of nifi version 1.16.3. I am > getting this error on start attempt: > > 2022-12-27

Workflows deployments across environments

Hello Nifi users, Question in the round. Let’s say I have 3 Nifi instances with dev, test and prod. What is the recommend way of deploying specific workflows from one environment to another ? We are using Nifi registry + GitHub as storage for the flows. What we want to achieve: * in

Re: Minifi and ssl config on NiFi

ssues you may run into. > > Regards, > Matt > > >> On Apr 17, 2022, at 11:40 AM, Jorge Machado wrote: >> >> I did this on the pass and I end up switching to Nifi. I think you should >> do the same. Minifi is kind of “Dead” not being developed anymore. I

Re: Minifi and ssl config on NiFi

I did this on the pass and I end up switching to Nifi. I think you should do the same. Minifi is kind of “Dead” not being developed anymore. I found better to just switch to single instance of nifi Regards Jorge > On 17. Apr 2022, at 03:30, David Early wrote: > > We are considering using

Re: Adding a node to a cluster

Hey, Are you using the zookeeper internally or an external one. I think you need to check at least two things: * Zookeeper connections. The node needs to register into zookeeper. * The node needs to connect to the cluster and replicate the flow files let’s say so. Overall scaling up is

Re: InvokeHTTP vs invalid SSL certificates

Just import the certificate into the trust store. > On 4. Mar 2022, at 13:59, Jean-Sebastien Vachon > wrote: > > Hi all, > > what is the best way to deal with invalid SSL certificates when trying to > open an URL using InvokeHTTP? > > > Thanks > > Jean-Sébastien Vachon > Co-Founder &

Re: Round robin load balancing eventually stops using all nodes

Are you are a dedicated port for transferring the data or using the http protocol ? I had similar issues with a remote port connection that got solved by not using the http protocol. > On 3. Sep 2021, at 14:13, Mike Thomsen wrote: > > We have a 5 node cluster, and sometimes I've noticed that

[jira] [Commented] (KAFKA-5164) SetSchemaMetadata does not replace the schemas in structs correctly

[ https://issues.apache.org/jira/browse/KAFKA-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391645#comment-17391645 ] Jorge Machado commented on KAFKA-5164: -- Actually not solved imho. See https://issues.apache.org/jira

[jira] [Commented] (KAFKA-7883) Add schema.namespace support to SetSchemaMetadata SMT in Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391583#comment-17391583 ] Jorge Machado commented on KAFKA-7883: -- I propose to have following:  new parameter named

[jira] [Commented] (KAFKA-7883) Add schema.namespace support to SetSchemaMetadata SMT in Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391536#comment-17391536 ] Jorge Machado commented on KAFKA-7883: -- Hey, I think this should not be marked as workaround

[jira] [Comment Edited] (KAFKA-7883) Add schema.namespace support to SetSchemaMetadata SMT in Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391536#comment-17391536 ] Jorge Machado edited comment on KAFKA-7883 at 8/2/21, 11:58 AM: Hey, I

Re: No Load Balancing since 1.13.2

Did you tried java 11 ? I have a client running a similar setup to yours but with a lower nigh version and it works fine. Maybe it is worth to try it. > On 27. Jul 2021, at 12:42, Axel Schwarz wrote: > > I did indeed, but I updated from u161 to u291, as this was the newest version > at that

Re: SocketTimeoutExceptions and JVM version

Is your input configured to be http or binary ? I can recommend using socket connection instead of https. You need an extra port but it could be worth. My last client we had similar issues. Ref: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html

[jira] [Commented] (MINIFI-546) Upgrade MiNiFi to Java 11

[ https://issues.apache.org/jira/browse/MINIFI-546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340560#comment-17340560 ] Jorge Machado commented on MINIFI-546: -- we actually stopped using the minifi because of the lacking

Re: RecordPath...With INNER records

Hey Greene, The LookupRecord as a RecordPath as input. Check out this docs: https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html#structure or https://www.nifi.rocks/record-path-cheat-sheet/

[jira] [Commented] (AIRFLOW-5655) Incorrect capitalization of env var causes task start to fail

[ https://issues.apache.org/jira/browse/AIRFLOW-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296355#comment-17296355 ] Jorge Machado commented on AIRFLOW-5655: I can confirm that I had the same error. Fixed

[jira] [Commented] (AIRFLOW-5655) Incorrect capitalization of env var causes task start to fail

[ https://issues.apache.org/jira/browse/AIRFLOW-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296337#comment-17296337 ] Jorge Machado commented on AIRFLOW-5655: I'm getting the same issue.  {code:java} File "

Re: NiFI fronend disconnects after some time

How much memory do you have on your instance? I would say the fetch size should be bigger then the Max Rows… > On 1. Mar 2021, at 06:37, Vibhath Ileperuma > wrote: > > Hi all, > > I tried to fetch a large dataset from postgresql using the ExcureSQL > processor. > The configurations I

Re: Feature requests for Mesos

Hi Samuel, To be honest, I would not invest any more Time on Mesos. The features from Kubernetes are just way better. :) > On 28. Feb 2021, at 12:54, Samuel Marks wrote: > > Decouple Apache ZooKeeper, enabling Apache Mesos to run completely without > ZooKeeper. Specifically enable a choice

Change internal state of a processor

Hello everyone, I think we need a tool that allows us to change the internal state of a processor. Like for example I would like to got 10 steps back from the state that has a value of 10. The resulting state would be 0 We could just overwrite what is there… Thanks

[jira] [Updated] (NIFI-8227) Enable extend of AbstractDatabaseFetchProcessor without adding the whole nifi-standard processors

[ https://issues.apache.org/jira/browse/NIFI-8227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated NIFI-8227: Affects Version/s: 1.12.1 > Enable extend of AbstractDatabaseFetchProcessor without add

[jira] [Updated] (NIFI-8227) Enable extend of AbstractDatabaseFetchProcessor without adding the whole nifi-standard processors

[ https://issues.apache.org/jira/browse/NIFI-8227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated NIFI-8227: Priority: Minor (was: Major) > Enable extend of AbstractDatabaseFetchProcessor without add

[jira] [Created] (NIFI-8227) Enable extend of AbstractDatabaseFetchProcessor without adding the whole nifi-standard processors

Jorge Machado created NIFI-8227: --- Summary: Enable extend of AbstractDatabaseFetchProcessor without adding the whole nifi-standard processors Key: NIFI-8227 URL: https://issues.apache.org/jira/browse/NIFI-8227

Re: Detect duplicate record reader

Hey Jeremy, Something linke this https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.DetectDuplicate/index.html

GenerateTableFetch Question

Hey again everyone, Is it possible that the GenerateTableFetch passes the max value into the table name field before it executes it ? Background for this I have a stored procedure that I pass in as table name. This needs two parameters a min and a max. Where min should be the maximum value

Re: How to proper use DistributedCacheServer ?

d.PutDistributedMapCache/index.html> > > --- > Chris Sampson > IT Consultant > chris.samp...@naimuri.com <mailto:chris.samp...@naimuri.com> > <https://www.naimuri.com/> > > > On Fri, 12 Feb 2021 at 14:48, Jorge Machado <mailto:jom...@me.com>>

How to proper use DistributedCacheServer ?

Hey everyone, Is there any documentation on how to use DistributedCacheServer ? Currently from what I see this is single point of failure or does it really sync the data between nodes ? I want to have something similar to zookeeper state but not in zookeeper because it needs to be

Re: [E] After upgrade to 1.11.4, flowController fails to start due to invalid clusterCoordinator port 0

For cluster mode check the configs that are on xml files. I had similar issues when I did not define the values. Letting them empty makes issues. Best regards Jorge CEO of Datamesh GmbH (www.dmesh.io) > On 9. Feb 2021, at 02:19, Pat White wrote: > > Thanks very much for the feedback Joe, much

Re: NIFI - Performance issues

Another thing to look would be to check if you are creating two much flow files as this hammers the disks. If you see very high memory usage it could be you are having to much data in attributes and not in the content of the flow file. > On 8. Feb 2021, at 07:07, nathan.engl...@bt.com wrote: >

[jira] [Created] (MINIFI-546) Upgrade minifi to java 11 and nifi 1.12

Jorge Machado created MINIFI-546: Summary: Upgrade minifi to java 11 and nifi 1.12 Key: MINIFI-546 URL: https://issues.apache.org/jira/browse/MINIFI-546 Project: Apache NiFi MiNiFi Issue

Re: Subject: [RESULT][VOTE] Release Apache Mesos 1.11.0 (rc1)

Unsubscribe > On 24. Nov 2020, at 15:22, Andrei Sekretenko wrote: > > Hi all, > > The vote for Mesos 1.11.0 (rc1) has passed with the > following votes. > > +1 (Binding) > -- > Vinod Kone > Till Toenshoff > Qian Zhang > Andrei Sekretenko > > There were no 0 or -1

[jira] [Commented] (AVRO-2890) java JSON decoder does not respect default values for fields

[ https://issues.apache.org/jira/browse/AVRO-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185861#comment-17185861 ] Jorge Machado commented on AVRO-2890: - I have the same issue > java JSON decoder does not resp

[jira] [Commented] (AVRO-1582) Json serialization of nullable fileds and fields with default values improvement.

[ https://issues.apache.org/jira/browse/AVRO-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185858#comment-17185858 ] Jorge Machado commented on AVRO-1582: - I think I'm hitting similar things with avro 1.10

Re: External Access using InvokeHTTP_Test processor and StandardSSLContextService

Hi Dan, Seems like this is a jvm issue. Try this: https://confluence.atlassian.com/kb/unable-to-connect-to-ssl-services-due-to-pkix-path-building-failed-error-779355358.html

Re: Urgent: HDFS processors throwing OOM - Compressed class space exception

How big are the files that you are trying to store? How much memory did you configure nifi ? > On 22. Jul 2020, at 06:13, Mohit Jain wrote: > > Hi team, > > I’ve been facing the issue while using any HDFS processor, e.g. - PutHDFS > throws the error - > Failed to write to HDFS due to

Re: Spark 3 pod template for the driver

Try to set spark.kubernetes.container.image > On 26. Jun 2020, at 14:58, Michel Sumbul wrote: > > Hi guys, > > I try to use Spark 3 on top of Kubernetes and to specify a pod template for > the driver. > > Here is my pod manifest or the driver and when I do a spark-submit with the > option:

Re: Using hadoop-cloud_2.12 jars

You can build it from source. Clone the spark git repo and run: ./build/mvn clean package -DskipTests -Phadoop-3.2 -Pkubernetes -Phadoop-cloud Regards > On 22. Jun 2020, at 11:00, Rahij Ramsharan wrote: > > Hello, > > I am trying to use the new S3 committers >

[jira] [Commented] (SPARK-31683) Make Prometheus output consistent with DropWizard 4.1 result

[ https://issues.apache.org/jira/browse/SPARK-31683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128625#comment-17128625 ] Jorge Machado commented on SPARK-31683: --- It would be great if we could use the RPC backend from

[jira] [Commented] (SPARK-31683) Make Prometheus output consistent with DropWizard 4.1 result

[ https://issues.apache.org/jira/browse/SPARK-31683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126671#comment-17126671 ] Jorge Machado commented on SPARK-31683: --- [~dongjoon] this only exporters the metrics from

Re: Arrow RecordBatches/Pandas Dataframes to (Arrow enabled) Spark Dataframe conversion in streaming fashion

Hey, from what I know you can try to Union them df.union(df2) Not sure if this is what you need > On 25. May 2020, at 13:53, Tanveer Ahmad - EWI wrote: > > Hi all, > > I need some help regarding Arrow RecordBatches/Pandas Dataframes to (Arrow > enabled) Spark Dataframe conversions. > Here

Re: Nifi - how to achieve a concurrent development and CI/CD

directly to master. > > > -Eric > > On Thu., May 14, 2020, 7:02 a.m. Jorge Machado, <mailto:jom...@me.com>> wrote: > Hi, > > Managing xml is always hard I think. Last time I need to do something similar > we used https://nifi.apache.org/registry.htm

Re: Nifi - how to achieve a concurrent development and CI/CD

Hi, Managing xml is always hard I think. Last time I need to do something similar we used https://nifi.apache.org/registry.html Works pretty well It was already 2 Years ago. Maybe now there is something better > On 14. May 2020, at 15:57, Michal Slama

Re: How to deal Schema Evolution with Dataset API

Ok, I found a way to solve it. Just pass the schema like this: val schema = Encoders.product[Person].schema spark.read.schema(schema).parquet(“input”)…. > On 9. May 2020, at 13:28, Jorge Machado wrote: > > Hello everyone, > > One question to the community. >

How to deal Schema Evolution with Dataset API

Hello everyone, One question to the community. Imagine I have this Case class Person(age: int) spark.read.parquet(“inputPath”).as[Person] After a few weeks of coding I change the class to: Case class Person(age: int, name: Option[String] = None) Then when I run

How to deal Schema Evolution with Dataset API

Hello everyone, One question to the community. Imagine I have this Case class Person(age: int) spark.read.parquet(“inputPath”).as[Person] After a few weeks of coding I change the class to: Case class Person(age: int, name: Option[String] = None) Then when I run

[jira] [Commented] (SPARK-26902) Support java.time.Instant as an external type of TimestampType

[ https://issues.apache.org/jira/browse/SPARK-26902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085759#comment-17085759 ] Jorge Machado commented on SPARK-26902: --- what about Supporting the interface Temporal ? > Supp

[jira] [Commented] (SPARK-30272) Remove usage of Guava that breaks in Guava 27

[ https://issues.apache.org/jira/browse/SPARK-30272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070774#comment-17070774 ] Jorge Machado commented on SPARK-30272: --- I failed to fix the guava stuff of course ... Today

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070519#comment-17070519 ] Jorge Machado commented on HADOOP-15669: I‘m using the docker images that the docker-image

[jira] [Commented] (SPARK-30272) Remove usage of Guava that breaks in Guava 27

[ https://issues.apache.org/jira/browse/SPARK-30272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070255#comment-17070255 ] Jorge Machado commented on SPARK-30272: --- So I was able to fix it. I build it with profile hadoop

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070223#comment-17070223 ] Jorge Machado commented on HADOOP-15669: Hi all,  I'm still seeing this "20/03/28 19:

[jira] [Comment Edited] (SPARK-30272) Remove usage of Guava that breaks in Guava 27

[ https://issues.apache.org/jira/browse/SPARK-30272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069933#comment-17069933 ] Jorge Machado edited comment on SPARK-30272 at 3/28/20, 4:25 PM: - Hey

[jira] [Commented] (SPARK-30272) Remove usage of Guava that breaks in Guava 27

[ https://issues.apache.org/jira/browse/SPARK-30272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069933#comment-17069933 ] Jorge Machado commented on SPARK-30272: --- Hey Sean,  This seems still to make problems for example

[jira] [Comment Edited] (SPARK-23897) Guava version

[ https://issues.apache.org/jira/browse/SPARK-23897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069352#comment-17069352 ] Jorge Machado edited comment on SPARK-23897 at 3/28/20, 9:50 AM: - I

[jira] [Commented] (SPARK-23897) Guava version

[ https://issues.apache.org/jira/browse/SPARK-23897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069352#comment-17069352 ] Jorge Machado commented on SPARK-23897: --- I think that master is actually broken at least

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046958#comment-17046958 ] Jorge Machado commented on SPARK-26412: --- Thanks for the Tipp. It helps > Allow Pandas

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046373#comment-17046373 ] Jorge Machado commented on SPARK-26412: --- Well I was thinking on something more. like I would like

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046265#comment-17046265 ] Jorge Machado commented on SPARK-26412: --- Hi, one question.  when using "a tuple of pd.S

[jira] [Commented] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034492#comment-17034492 ] Jorge Machado commented on SPARK-24615: --- Yeah, that was my question. Thanks for the response. I

[jira] [Commented] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034277#comment-17034277 ] Jorge Machado commented on SPARK-24615: --- [~tgraves] thanks for the input. It would be great

[jira] [Commented] (SPARK-30647) When creating a custom datasource File NotFoundExpection happens

[ https://issues.apache.org/jira/browse/SPARK-30647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029788#comment-17029788 ] Jorge Machado commented on SPARK-30647: --- 2.4x has the same issue. > When creating a cus

[jira] [Commented] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028856#comment-17028856 ] Jorge Machado commented on SPARK-27990: --- [~nchammas]: Just pass this like:  {code:java} //.option

[jira] [Commented] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028854#comment-17028854 ] Jorge Machado commented on SPARK-27990: --- Can we backport this to 2.4.4 ? > Provide a

[jira] [Commented] (SPARK-30647) When creating a custom datasource File NotFoundExpection happens

[ https://issues.apache.org/jira/browse/SPARK-30647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024242#comment-17024242 ] Jorge Machado commented on SPARK-30647: --- I found a way to overcome this. I just replace %20

[jira] [Comment Edited] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023935#comment-17023935 ] Jorge Machado edited comment on SPARK-23148 at 1/27/20 7:22 AM: Hi

[jira] [Updated] (SPARK-30647) When creating a custom datasource File NotFoundExpection happens

[ https://issues.apache.org/jira/browse/SPARK-30647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-30647: -- Issue Type: Bug (was: Improvement) > When creating a custom datasource F

[jira] [Created] (SPARK-30647) When creating a custom datasource File NotFoundExpection happens

Jorge Machado created SPARK-30647: - Summary: When creating a custom datasource File NotFoundExpection happens Key: SPARK-30647 URL: https://issues.apache.org/jira/browse/SPARK-30647 Project: Spark

[jira] [Comment Edited] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023935#comment-17023935 ] Jorge Machado edited comment on SPARK-23148 at 1/26/20 9:29 PM: Hi

[jira] [Comment Edited] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023935#comment-17023935 ] Jorge Machado edited comment on SPARK-23148 at 1/26/20 9:29 PM: Hi

[jira] [Commented] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023935#comment-17023935 ] Jorge Machado commented on SPARK-23148: --- So I have the same problem if I create a custom data

[jira] [Commented] (SPARK-29158) Expose SerializableConfiguration for DSv2

[ https://issues.apache.org/jira/browse/SPARK-29158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997947#comment-16997947 ] Jorge Machado commented on SPARK-29158: --- How can we get SerializableConfiguration with 2.4.4 ? Any

[jira] [Created] (AIRFLOW-6132) AzureContainerInstancesOperator should allow to pass in tags

Jorge Machado created AIRFLOW-6132: -- Summary: AzureContainerInstancesOperator should allow to pass in tags Key: AIRFLOW-6132 URL: https://issues.apache.org/jira/browse/AIRFLOW-6132 Project: Apache

[jira] [Commented] (HDFS-9924) [umbrella] Nonblocking HDFS Access

[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900426#comment-16900426 ] Jorge Machado commented on HDFS-9924: - Any ideas how to improve the hdfs dfs put command with async io

[jira] [Commented] (HDFS-916) Rewrite DFSOutputStream to use a single thread with NIO

[ https://issues.apache.org/jira/browse/HDFS-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900421#comment-16900421 ] Jorge Machado commented on HDFS-916: Hi Guys, I know this is pretty old but is there any status

Restarting mesas-agent kills executors

/store/docker \ --gc_delay=3weeks \ --attributes= KillMode=control-cgroup Restart=always RestartSec=20 LimitNOFILE=infinity CPUAccounting=true MemoryAccounting=true TasksMax=infinity [Install] WantedBy=multi-user.target Any tipp ? thx Jorge Machado www.jmachado.me

[jira] [Commented] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882686#comment-16882686 ] Jorge Machado commented on SPARK-24615: --- Hi Guys, is there any progress here ? I would like

How to run spark on GPUs

Hi Guys, what is the current recommend way to use GPUs on spark ? Which scheduler should we use ? Mesos Or Kubernetes ? What are the approaches to follow until https://issues.apache.org/jira/browse/SPARK-24615 is in place. Thanks Jorge

Re: [VOTE] Release Apache Mesos 1.8.0 (rc3)

error > message you describe sounds to me more like a build issue, i.e. it sounds > like the version of the nvidia driver is different between the docker image > and the host system? > > Maybe you could continue investigating to see if this is related to the > release itse

Re: [VOTE] Release Apache Mesos 1.8.0 (rc3)

Hi all, did someone tested it on ubuntu 18.04 + nvidia-docker2 ? We are having some issues using the cuda 10+ images when doing real processing. We still need to check some things but basically we get: kernel version 418.56.0 does not match DSO version 410.48.0 -- cannot find working devices

[jira] [Commented] (MESOS-9740) Invalid protobuf unions in ExecutorInfo::ContainerInfo will prevent agents from reregistering with 1.8+ masters

[ https://issues.apache.org/jira/browse/MESOS-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824829#comment-16824829 ] Jorge Machado commented on MESOS-9740: -- we are running mesos 1.7.1 and have one slave with ubuntu

Re: Mesos on ssl

1.8 will be the first official release supporting SSL on Ubuntu 18.04. > > That said, I'm not sure what you encountered is exactly the same bug that > caused the Mesos tests to fail though. Just a guess ;) > > On Fri, Apr 5, 2019, 12:58 AM Jorge Machado wrote: > >> Hi G

Mesos on ssl

endpoint does not work and it just hangs. No logs nothing... I'm testing this on ubuntu 18.04. Any tipps ? thanks Jorge Jorge Machado www.jmachado.me

[jira] [Commented] (MESOS-6851) make install fails the second time

[ https://issues.apache.org/jira/browse/MESOS-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808550#comment-16808550 ] Jorge Machado commented on MESOS-6851: -- ah nice link. I see that the first one is for Centos anyone

Why does not mesos provide linux packages ?

Hi Guys, why don't we have packages for the main ubuntu distributions ? like ubuntu and redhat ? I have the feeling that everyone is building and creating this packages to distribute. Is there a way that we could improve this ? thanks

[jira] [Comment Edited] (MESOS-6851) make install fails the second time

[ https://issues.apache.org/jira/browse/MESOS-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808527#comment-16808527 ] Jorge Machado edited comment on MESOS-6851 at 4/3/19 9:32 AM: -- So I try

[jira] [Commented] (MESOS-6851) make install fails the second time

[ https://issues.apache.org/jira/browse/MESOS-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808527#comment-16808527 ] Jorge Machado commented on MESOS-6851: -- So I try it right now and it does not fit. It would much

[jira] [Commented] (MESOS-6851) make install fails the second time

[ https://issues.apache.org/jira/browse/MESOS-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808477#comment-16808477 ] Jorge Machado commented on MESOS-6851: -- same problem here. I just checked for master and I cannot

[jira] [Commented] (SPARK-27208) RestSubmissionClient only supports http

[ https://issues.apache.org/jira/browse/SPARK-27208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806495#comment-16806495 ] Jorge Machado commented on SPARK-27208: --- Any help here please ? I really think this is broken

Re: [MESOS-8248] - Expose information about GPU assigned to a task

another way would be to just use cadvisor > On 22 Mar 2019, at 08:35, Jorge Machado wrote: > > Hi Mesos devs, > > In our use case from mesos we need to get gpu resource usage per task and > build dashboards on grafana for it. Getting the metrics to Grafana we will

[MESOS-8248] - Expose information about GPU assigned to a task

it in the NvidiaGpuIsolatorProcess and get the metrics via the host. Anything more that I should check ? Thanks a lot Jorge Machado www.jmachado.me

  1   2   3   4   >