Re: [VOTE] Release Apache Tez 0.5.0 rc2

2014-08-31 Thread Jonathan Eagles
+1 (non-binding). Verified signatures and checksums. Built from source and successfully ran several examples on a test system On Mon, Sep 1, 2014 at 12:45 AM, Siddharth Seth wrote: > +1 (binding). > > Verified signatures and checksums. > Built from source and successfully ran several examples

Re: Swimlanes Usage

2014-09-16 Thread Jonathan Eagles
Thad, the swimlanes tool may need some added robustness for the case above. Can you capture the yarn logs for this job to help in debugging? jeagles On Tue, Sep 16, 2014 at 8:46 PM, Thaddeus Diamond < thaddeus.diam...@gmail.com> wrote: > Hey guys, > > I'm trying to use the swimlanes tool (very n

Re: [VOTE] Release Apache Tez 0.5.1 rc0

2014-10-02 Thread Jonathan Eagles
+1 (non-binding). I was able to verify this build is working successfully. On Sun, Sep 28, 2014 at 9:41 PM, Bikas Saha wrote: > Folks, > > > > I have created an Apache Tez 0.5.1 release candidate (rc0). > > > > This is an incremental release that contains an important fix for API > consistency t

Re: [ANNOUNCE] New Apache Tez Committer - Jeff Zhang

2014-10-16 Thread Jonathan Eagles
Great work, Jeff! On Oct 16, 2014 6:20 PM, "Jianfeng (Jeff) Zhang" wrote: > Thanks everyone for your help, your comment, review do help me a lot to > get familiar with tez project. I am so excited to become tez committer, > will continue contribute to tez. [?] > > Best Regards, > Jeff Zhang > >

Re: [VOTE] Release Apache Tez 0.5.2 rc0

2014-11-07 Thread Jonathan Eagles
FYI: Found that local mode is broken when this build is compiled against 2.5.x and run against 2.6. Error seems to stem from the private unstable ContainedId.newInstance api change from YARN-2229. A recompile will fix this issue, but wanted to bring this up since we want to consider adding an overl

Re: Tez TimelineServer

2014-11-11 Thread Jonathan Eagles
Great, Dharmesh. Make sure to include the web address you are trying to browse (for example localhost:8188/ws/v1/timeline/TEZ_DAG_ID). That will help in debugging. Jon On Tue, Nov 11, 2014 at 10:17 AM, Dharmesh Kakadia wrote: > I will file a JIRA with log attached. > > Currently I am only lookin

Re: [DISCUSS] What versions of hadoop should we support?

2014-12-11 Thread Jonathan Eagles
Adding to the discussion. I am using some 2.5 and 2.6 On Wed, Dec 10, 2014 at 8:02 PM, Cheolsoo Park wrote: > We're using EMR 2.4 in production, and it will take a while to upgrade to > 2.6 depending on how fast EMR can provide us 2.6 AMI. But if that's what we > need to fully use Tez-8, we will

[ANNOUNCE] Apache Tez 0.6.0.

2015-01-30 Thread Jonathan Eagles
The Apache Tez team is proud to announce the release of Apache Tez version 0.6.0 The Apache Tez project is aimed at creating a framework to build efficient and scalable data processing applications that can be modeled as data flow graphs. This is an feature release that contains the first impleme

Re: [ANNOUNCE] New Apache Tez Committer - Sreenath Somarajapuram

2015-05-13 Thread Jonathan Eagles
Congratulations, Sreenath! On Wed, May 13, 2015 at 5:45 PM, Jeff Zhang wrote: > Congratulations, Seenath. > > > > On Thu, May 14, 2015 at 4:09 AM, Hitesh Shah wrote: >> >> Hi all, >> >> I am very pleased to announce that Sreenath Somarajapuram has been voted >> in as a committer in the Apache Te

Re: tez 0.7.0 released?

2015-05-19 Thread Jonathan Eagles
Michael, Your question is very timely as we are currently putting the final touches on both the 0.6.1 and 0.7.0 releases. You can expect to see the announce email regarding theses releases on the users list soon if not today. Jon On Tue, May 19, 2015 at 10:43 AM, Michael Howard wrote: > I have

[ANNOUNCE] Apache Tez 0.6.1

2015-05-20 Thread Jonathan Eagles
The Apache Tez team is proud to announce the release of Apache Tez version 0.6.1 The Apache Tez project is aimed at creating a framework to build efficient and scalable data processing applications that can be modeled as data flow graphs. This release contains all the bug fixed and improvement ma

Re: Flaky queries with 'TezSession has already shut down'

2015-10-04 Thread Jonathan Eagles
What's happening here is that the Tez AM has a runtime dependency on jackson libraries due to ATSHistoryLoggingService (due to TimelineClientImpl in YARN). It looks like your application is shipping a different version of jackson that YARN was expecting. Can you check what two versions of jackson l

Re: Tez 0.8.4 building failed

2016-08-24 Thread Jonathan Eagles
Thanks for report this. I have sometimes seen this issue when switching git branches. I don't know if there is a JIRA tracking this issue or not but I have found that it can resolved this by removing the npm related build artifacts and then rebuilding. For example from the root directory of the te

Re: Re: Tez 0.8.4 building failed

2016-08-25 Thread Jonathan Eagles
: > Hello Jonathan > > Thank you for you answer . But I download the source release from > tez.apache.org . It's not a git repo . > > > At 2016-08-25 14:02:46, "Jonathan Eagles" wrote: > > Thanks for report this. I have sometimes seen this issue when switching

Re: Zip Exception since commit da4098b9

2016-09-28 Thread Jonathan Eagles
The RM will need CORS support turned on so that the Tez UI can query the live Tez AM from the Tez UI. This support was added in Hadoop 2.7.2. https://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-common/HttpAuthentication.html On Wed, Sep 28, 2016 at 6:28 PM, Madhusudan Ramanna wrot

Re: Question about yarn.timeline-service.enabled

2016-12-07 Thread Jonathan Eagles
Adding to Gopal, this configuration can be found as part of the Tez UI setup which has more information regarding the configuration and enablement of the Tez UI that is powered from the data stored in the Timeline store. http://tez.apache.org/tez-ui.html Regards, jeagles On Wed, Dec 7, 2016 at 9

Re: [DISCUSS] Miniumum hadoop version on master to 2.7

2017-04-19 Thread Jonathan Eagles
+1. I submitted a patch to upgrade to 2.6.3 to fix a port binding issues in tests. An upgrade to 2.7 will fix this. https://issues.apache.org/jira/browse/TEZ-3622 On Wed, Apr 19, 2017 at 3:22 PM, Siddharth Seth wrote: > Assuming the lac of a response to indicate there's no objections to making >

Re: Running commands on Hive on Tez failed with NPE

2018-02-13 Thread Jonathan Eagles
Silas, thanks for reaching out to the Tez community. Going by the stack trace, we have not yet reach tez code. The best way to get some effective help is by reaching out to the hive user list. Jon On Mon, Feb 12, 2018 at 10:55 PM, Ge Silas wrote: > Hi Tez community, > > I recently encountered w

Re: Tez jobs created by Sqoop are getting killed

2018-02-14 Thread Jonathan Eagles
Thanks for reporting this. The best way to diagnose this is to post the AM logs. The client logs show there was an error connecting to the application master (AM) process. Likely it was unable to startup correctly, and the AM logs will show why. Regards, Jon On Feb 14, 2018 12:21 AM, "Snehal B"

Re: Tez not returning .json results inspite of request completing sucessfully

2018-03-19 Thread Jonathan Eagles
Thanks for reaching out to us, Jyotsna. Let me make sure I understand what your setup and the issue you are experiencing. It sounds like the setup looks like this. Browser -> Third party service "thirdeye-services" -> HiveServer2 -> Tez Since you are showing logs and screenshots of the success

Re: Tez not returning .json results inspite of request completing sucessfully

2018-03-19 Thread Jonathan Eagles
gurations for tez that I might have to look for? > > > > On Mon, Mar 19, 2018, 21:37 Jonathan Eagles wrote: > >> Thanks for reaching out to us, Jyotsna. >> >> Let me make sure I understand what your setup and the issue you are >> experiencing. It sounds lik

Re: Tez not returning .json results inspite of request completing sucessfully

2018-03-20 Thread Jonathan Eagles
ent (368802354) connection to slc06ffs/ >> 10.245.175.89:55310 from thirdeye]: DEBUG ipc.Client: IPC Client >> (368802354) connection to slc06ffs/10.245.175.89:55310 from thirdeye got >> value #1814 >> 18/03/19 23:48:33 [HiveServer2-Handler-Pool: Thread-27]: DEBUG >&g

Re: Regarding Hanging Tez Queries in Nifi

2019-04-25 Thread Jonathan Eagles
Varun, Thanks for reaching out about this issue. With no experience direct experience with NiFi, my help may not be fully accurate. However, here is what could be going on. Hive queries can launch Tez in session mode and non-session mode. Non-session mode execution is simple 1) launch an AM (applic

Re: Tez -0.9.2 build errror in windows 10

2019-11-06 Thread Jonathan Eagles
I will have to look at the failure in more detail to see why the Tez UI isn't building correctly. In the meantime, you can disable the Tez UI in the build to see if there are other errors with building in Windows. So modifying the command you provided above. mvn clean package -DskipTests -Dhadoop

Re: Tez -0.9.2 build errror in windows 10

2019-11-11 Thread Jonathan Eagles
Didn't hear back from you, so emailing you directly in case you aren't on the user list. See the comments below for answers to your questions. On Wed, Nov 6, 2019 at 11:07 AM Jonathan Eagles wrote: > I will have to look at the failure in more detail to see why the Tez UI >

Re: Tez job vertex failed due to InvalidProtocolBufferException: Protocol message was too large

2020-04-10 Thread Jonathan Eagles
Thanks for reporting this issue with deserialization of Configuration objects greater than 32MB. I have filed a JIRA on your behalf https://issues.apache.org/jira/browse/TEZ-4142 which can be used to track this change in the Tez project. I have created a patch request to the 0.10.x and 0.9.x lines

Re: A mapper task in Tez is getting a lot more data than other mappers

2020-09-23 Thread Jonathan Eagles
Without much to go on, I assume this is a hive query. I am not a hive expert, but I can try to give some help. The hive mailing list will surely give a better answer. What is the input format? What is the hive strategy used? If orc is used, hive can be used to give statistics about the stripes in

Re: Porting legacy MapReduce application to Tez

2020-12-17 Thread Jonathan Eagles
This is what I use in production that has many benefits. In this case mapreduce.application.framework.path is the runtime classpath tar.gz file that is custom built mapreduce runtime environment, perhaps similar to nutch 1) localizing one tar.gz file instead of many individual jars 2) minimal jar h

Re: Tez + YARN ATSv2

2020-12-22 Thread Jonathan Eagles
This feature was in progress for some time but progress was halted. https://issues.apache.org/jira/plugins/servlet/mobile#issue/TEZ-3820 Many of the stakeholders can be found on that ticket The issue regarding scale depends on usage. I run atsv1.5 and write 300,000 history files a day. But they

Re: Setting up Tez on windows

2021-06-12 Thread Jonathan Eagles
This request will be better directed to the hive mailing list to get the answer you need. On Sat, Jun 12, 2021, 3:13 PM tar chan wrote: > Hi there, > > I am using Embedded mode of HiveLocalServer2 setup on windows machine. > > I setup'ed using java files like HiveLocalServer2 , HiveConf, > Hive

Re: Setting up Tez on windows

2021-06-12 Thread Jonathan Eagles
On Sat, Jun 12, 2021, 3:15 PM Jonathan Eagles wrote: > This request will be better directed to the hive mailing list to get the > answer you need. > > On Sat, Jun 12, 2021, 3:13 PM tar chan wrote: > >> Hi there, >> >> I am using Embedded mode of HiveLoca

Re: [DISCUSS] Tez 0.10.4 Release Planning

2024-06-28 Thread Jonathan Eagles
Data-via-events for ordered fetch https://issues.apache.org/jira/browse/TEZ-4570 This actually represents a very significant bug as Pig fetches unordered output using ordered fetch. Now that data-via-events is now the default, it means that Pig fails with Tez out of the box. Jonathan On Fri, Jun