Re: One partitions segment files in different log-dirs

Hi Thank you for a answer. The answer I was looking for was how partition segment files are distributed over kafka-logs directories? In example I have one broker with two log directories: kafka-logs1 and kafka-logs2 both of them 100MB in example and partition file segment size is 90MB. If

Re: Index population over table contains 2.3 x 10^10 records

ooks similar to problems I've seen many years ago around > mapreduce.task.io.sort.mb. You can try reducing that value. It also may be > related to a bug in your Hadoop version. > > Good luck! > > On 3/22/18 4:37 AM, Margusja wrote: >> Hi >> Needed to recreate

Index population over table contains 2.3 x 10^10 records

Hi Needed to recreate indexes over main table contains more than 2.3 x 10^10 records. I used ASYNC and org.apache.phoenix.mapreduce.index.IndexTool One index succeed but another gives stack: 2018-03-20 13:23:16,723 FATAL [IPC Server handler 0 on 43926]

Re: Hiveserver2 - Beeline How to enable progress bar (Tez)

Does it depends what kind of engine you are using - tez versus mr? Br margus > On 1 Feb 2018, at 17:57, Shankar Mane wrote: > > Any update? > > I m not getting progress bar with beeline client. tried both embedded and > remote connection. > > It's working properly with

Re: Spark 2.1.1 (Scala 2.11.8) write to Phoenix 4.7 (HBase 1.1.2)

Also I see that Logging is moved to internal/Logging. But is there package for my environment I can use? Margus > On 30 Jan 2018, at 17:00, Margusja <mar...@roo.ee> wrote: > > Hi > > Followed page (https://phoenix.apache.org/phoenix_spark.html > <

Spark 2.1.1 (Scala 2.11.8) write to Phoenix 4.7 (HBase 1.1.2)

Hi Followed page (https://phoenix.apache.org/phoenix_spark.html >) and trying to save to phoenix. Using spark-1.6.3 it is successful but using

Spark 2.1.1 (Scala 2.11.8) write to Phoenix 4.7 (HBase 1.1.2)

Hi Followed page (https://phoenix.apache.org/phoenix_spark.html ) and trying to save to phoenix. Using spark-1.6.3 it is successful but using spark-2.1.1 it is not. First error I am getting using spark-2.1.1 is that: Error:scalac: missing or

Get broadcast (set in one method) in another method

Hi Maybe I am overthinking. I’d like to set broadcast in object A method y and get it in object A method x. In example: object A { def main (args: Array[String]) { y() x() } def x() : Unit = { val a = bcA.value ... } def y(): String = { val bcA =

Re: run spark job in yarn cluster mode as specified user

Hi org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor requires user in each node and right permissions set in necessary directories. Br Margus > On 22 Jan 2018, at 13:41, sd wang wrote: > >

Re: run spark job in yarn cluster mode as specified user

Hi One way to get it is use YARN configuration parameter - yarn.nodemanager.container-executor.class. By default it is org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor - gives you user who executes script. Br

Re: ambari 2.6.0 hang up after Registering your hosts

Maybe ambari-database somehow get broken? Margus Roo > On 10 Nov 2017, at 05:11, lk_hadoop wrote: > >

Re: spark job paused(active stages finished)

You have to deal with failed jobs. In example try catch in your code. Br Margus Roo > On 9 Nov 2017, at 05:37, bing...@iflytek.com wrote: > > Dear,All > I have a simple spark job, as below, all tasks in the stage 2(sth failed, > retry) already finished. But the next stage never run. > >

Re: Developing plugin using 0.7.0 and RANGER-1479

ty. So it was logical not to build new audit framework but use existing - Ranger. Used examples from git and development process was clear and easy. Br Margus > On 7 Nov 2017, at 09:00, Margusja <mar...@roo.ee> wrote: > > Hi > > We hit https://issues.apache.org/jira/b

Developing plugin using 0.7.0 and RANGER-1479

Hi We hit https://issues.apache.org/jira/browse/RANGER-1479 . At the moment we use Ranger-0.7.0 from Hortonworks distro (HDP-2.6.1.0). What are my option at the moment? 1 - download latest src and compile it and use it to build our custom

Re: Authenticate from SQL

Hi In one old project where usernames and passwords are in RDB, we need authenticate users from RDB before they can go via REST to HBase. So the first thing was Knox. Best regards, Margus Roo skype: margusja phone: +372 51 48 780 web: http://margus.roo.ee On 04/11/14 04:02, Bharath

Authenticate from SQL

Hi I am looking solutions where users before using HBase rest will be authenticate from SQL (in example from Oracle). Is there any best practices or ready solutions for HBase? -- Best regards, Margus Roo skype: margusja phone: +372 51 48 780 web: http://margus.roo.ee

[no subject]

unsubscribe - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org

The filesystem under path '/' has n CORRUPT files

not exist As I understand There is nothing to do. Tried to delete it [hdfs@bigdata1 dfs]$ hdfs dfs -rm /user/hue/cdr/2014/12/10/table10.csv rm: `/user/hue/cdr/2014/12/10/table10.csv': No such file or directory So what sould I do? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http

ApplicationMaster link on cluster web page does not work

there is nothing interesting -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)

Re: ApplicationMaster link on cluster web page does not work

:90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:47321 :::90.190.106.33:8088 ESTABLISHED 15723/java ... ... ... wtf? Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h

Re: ApplicationMaster link on cluster web page does not work

Moved resourcemanager to another server and it works. I guess I have some network miss routing there :) Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314

Re: Worker dies (bolt)

/var/lib/storm/workers/*/pids [root@dlvm2 sysconfig]# kill -9 4179 after it new worker is coming up and it works again around 2500 cycles and stops and I have to kill pid again. Any ideas? Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo

Re: Worker dies (bolt)

: KafkaConsumerSpout default Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) On 03/06/14 09:58, Margusja wrote: Hei I have made a new test and discovered that in my

Re: Worker dies (bolt)

(); 75 } And now it is running. Strange because when worker died then I see log rows from spout. But I think it is related somehow with the internal stuff in storm. Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype

Worker dies (bolt)

, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)

Exception failure: java.lang.ClassNotFoundException: org.apache.spark.streaming.kafka.KafkaReceiver

]# Any ideas? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)

Re: Announcing Spark 1.0.0

Now I can download. Thanks. Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) On 30/05/14 13:48, Patrick Wendell wrote: It is updated - try holding Shift

CSharp librari and Producer Closing socket for because of error (kafka.network.Processor),java.nio.BufferUnderflowException

can I give to producer brokers list to get fault tolerance in case one broker is down? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)

Re: Kafka producer in CSharp

Thank you for response. I think HTTP is ok. I have two more question in case of HTTP. 1. Can I have fault tolerance in case I have two or more brokers? 2. Can I ack that message is in queue? Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com

Re: Kafka producer in CSharp

-http-endpoint is up? And if not up then using another? Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) On 13/05/14 10:49, Margusja wrote: Thank you

Kafka producer in CSharp

version (kafka_2.9.1-0.8.1.1) so I downloaded kafka-0.7.1-incubating. Now I was able to send messages using CSharp code. So is there workaround how I can use latest kafka version and CSharp ? Or What is the latest kafka version supporting CSharp producer? -- Best regards, Margus (Margusja) Roo +372

Re: ZooKeeper available but no active master location found

Yes there is: groupIdorg.apache.hbase/groupId artifactIdhbase/artifactId version0.92.1/version Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber

This server is in the failed servers list

@sandbox ~]# cat /etc/hosts 127.0.0.1 localhost.localdomain localhost 10.0.2.15 sandbox.hortonworks.com sandbox Any hints? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE

Re: This server is in the failed servers list

= mytable Table = simple_hcat_load_table Table = users Table = weblogs Maybe my mistake helps somebody :) Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) On 10

ZooKeeper available but no active master location found

resolving. Any hints? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)

NodeHealthReport local-dirs turned bad

? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)

Re: NodeHealthReport local-dirs turned bad

tnx got it work. In my init script I used wrong user. It was permissions problem like Rohith said. Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) On 19/03

Command line vector to sequence file

dfs -text output2/part-m-0 gives me something like: /data.txt1,1 2,1 1,2 2,2 3,3 8,8 8,9 9,8 9,9 and that is not sequence file format as I understand. I know there are java API but I am looking command line. -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee

Re: Command line vector to sequence file

Thank you, I am going to try it. Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) -BEGIN PUBLIC KEY

java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected

704f1e463ebc4fb89353011407e965 This command was run using /usr/lib/hadoop/hadoop-common-2.2.0.2.0.6.0-101.jar [speech@h14 ~]$ Any hint what I am doing wrong? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x

Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields

am doing something wrong :) Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) -BEGIN PUBLIC KEY

Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields

Okey sorry for the mess [speech@h14 mahout]$ /usr/share/apache-maven/bin/mvn -DskipTests clean install -Dhadoop2.version=2.2.0 - did the trick Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h

Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Hi, is there any information about the problem I submitted? Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) -BEGIN PUBLIC KEY

Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) [hduser@vm38 ~]$ Tervitades, Margus (Margusja

Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

.jar pi 2 5 works. -- Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE

Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

) Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE

Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields

.../job.xml there were values from sourceyarn-default.xml/source even I set them in yarn-site.xml. Typical mess up :) Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314

class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields

) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1343) Where to start digging? -- Tervitades, Margus (Margusja) Roo +372 51 48

Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields

:13:21,165 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: Scheduling Log Deletion for application: application_1393848686226_0019, with delay of 10800 seconds ... Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http

unsubscribe

Task failure in slave node

getting errors when I run command from master? -- Regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee skype: margusja -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl

Re: Task failure in slave node

Than you, it resolved the problem. Funny, I don't remember that I copied mahout libs to n1 hadoop but there they are. Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee skype: margusja -BEGIN PUBLIC KEY

[Components] Node activates less conditional outgoing nodes than required

( 25000 = 99 )] node9 - node10 node10 - node2 node11 - node10 node12 - node10 node13 - node10 node14 - node10 node15 - node6 node16 - node6 } -- Regards Margusja Phone: +372 51 48 780 MSN: margu...@kodila.ee skype: margusja web: http://margusja.pri.ee -- Components mailing list Components

[Components] Uncaught exception 'ezcWorkflowInvalidInputException' with message 'children == '

: #0 /var/www/arendus/margusja/ecwf/demowf_continue.php(41): ezcWorkflowExecution-resume(Array) #1 {main} thrown in /usr/share/pear/ezc/Workflow/interfaces/execution.php on line 359 As much I understand in class ezcWorkflowConditionIsEqual extends ezcWorkflowConditionComparison method evaluate

[suPHP] terminate called after throwing an instance of 'suPHP::LookupException'

20:07 users drwxrwxr-x 3 root root 4.0K 2008-09-03 21:21 vhosts Ok that is not a problem but can someone describe what is changed? --- Margus Margusja Roo +3725148780 skype: margusja msn: margusja at kodila.ee homepage: http://margusja.pri.ee Margusja wrote: Hello. I still use

Re: Timeout: No Response from client-server

13 packets received by filter 0 packets dropped by kernel 10:32:27 root[load: 0@client-server ~# Best regards, Margus Margusja Roo +3725148780 skype: margusja msn: margu...@kodila.ee homepage: http://margusja.pri.ee Margusja wrote: Hi I have two server-rooms (DZ-A and DZ-B) in DZ

Re: Timeout: No Response from client-server

client-server. 13:51:03 root[load: 0@monitoring-server ~# Best regards, Margus Margusja Roo +3725148780 skype: margusja msn: margu...@kodila.ee homepage: http://margusja.pri.ee Dave Shield wrote: What is the output of running snmpgetnext -v1 -c sw -d client-server system

Re: Timeout: No Response from client-server

Found a solution. Big my mistake. There were firewall closed from client-server to master-server Best regards, Margus Margusja Roo +3725148780 skype: margusja msn: margu...@kodila.ee homepage: http://margusja.pri.ee Margusja wrote: 13:50:48 root[load: 1@monitoring-server ~# snmpgetnext

Timeout: No Response from client-server

noauthexact allallnone SNMPD is running: root 24039 0.0 0.8 8992 4248 ?S May06 0:05 /usr/sbin/snmpd -Lsd -p /var/run/snmpd.pid and in both servers: NET-SNMP version: 5.4.2.1 --- Any ideas? -- Best regards, Margus Margusja Roo +3725148780 skype

[Fedora-xen] Problem to using virt-install

) Domain-0 0 489 1 r- 738.2 -- --- Margusja +3725148780 skype: margusja msn: [EMAIL PROTECTED] homepage: http://margusja.pri.ee -- Fedora-xen mailing list Fedora-xen@redhat.com https://www.redhat.com/mailman/listinfo/fedora-xen

SMS reciving problem

that all is ok in their side. Best regards, Margusja --- Margusja +37251780 skype: margusja msn: [EMAIL PROTECTED] homepage: http://margusja.pri.ee --- Margusja +37251780 skype: margusja msn: [EMAIL PROTECTED] homepage: http://margusja.pri.ee

Re: SMS reciving problem

ci3-software-1 = 1301 --- Margusja +37251780 skype: margusja msn: [EMAIL PROTECTED] homepage: http://margusja.pri.ee Margusja wrote: I use kannel: Kannel bearerbox version `1.4.1'. Build `Oct 16 2006 10:48:53', compiler `4.1.1 20060525 (Red Hat 4.1.1-1)'. System Linux, release 2.6.19

sms reciving problem

that all is ok in their side. Best regards, Margusja --- Margusja +37251780 skype: margusja msn: [EMAIL PROTECTED] homepage: http://margusja.pri.ee

[GENERAL] query is wery slow with _t() function

nextval('public.sys_txt_code_id_seq'::text) code | character varying(100) | descr| character varying(255) | group_id | integer[] | code_new | character varying(100) | Indexes: sys_txt_code__code_ukey unique, btree (code) sys_txt_code_ukey unique, btree (id) Reg, Margusja