kafka + mysql filtering problem

2016-02-29 Thread franco barrientos
spc.broadcast(x.split(",")(5).filterNot(toRemove)) } } var msg = rule_apply(1, mto, rules) var word = lines.map(x => msg) word.print() ssc.start() ssc.awaitTermination() } } The problem is that mto variable always returns to “0” value after mapping lines DStream. I

TF-IDF Question

2015-06-04 Thread franco barrientos
]²)?. Regards and thanks in advance. Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 franco.barrien...@exalitica.com mailto:franco.barrien...@exalitica.com www.exalitica.com http://www.exalitica.com/

null Error in ALS model predict

2014-12-24 Thread Franco Barrientos
a ratings.first() I get the follow error: Why this happend? I need to use this second way. Thanks in advance, Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 mailto:franco.barrien...@exalitica.com franco.barrien

RE: Effects problems in logistic regression

2014-12-22 Thread Franco Barrientos
Thanks again DB Tsai, LogisticRegressionWithLBFGS works for me! De: Franco Barrientos [mailto:franco.barrien...@exalitica.com] Enviado el: jueves, 18 de diciembre de 2014 16:42 Para: 'DB Tsai' CC: 'Sean Owen'; user@spark.apache.org Asunto: RE: Effects problems in logistic regression

Effects problems in logistic regression

2014-12-18 Thread Franco Barrientos
calculates exp(-1*(-0.4021+(-207.1749)*amount)) this is a big number, in fact infinity for spark. How can I treat this variable? or why this happened? Thanks , Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893

RE: Effects problems in logistic regression

2014-12-18 Thread Franco Barrientos
Thanks I will try. De: DB Tsai [mailto:dbt...@dbtsai.com] Enviado el: jueves, 18 de diciembre de 2014 16:24 Para: Franco Barrientos CC: Sean Owen; user@spark.apache.org Asunto: Re: Effects problems in logistic regression Can you try LogisticRegressionWithLBFGS? I verified

Percentile

2014-11-27 Thread Franco Barrientos
Hi folks!, Anyone known how can I calculate for each elements of a variable in a RDD its percentile? I tried to calculate trough Spark SQL with subqueries but I think that is imposible in Spark SQL. Any idea will be welcome. Thanks in advance, Franco Barrientos Data Scientist Málaga

join 2 tables

2014-11-12 Thread Franco Barrientos
: Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 mailto:franco.barrien...@exalitica.com franco.barrien...@exalitica.com http://www.exalitica.com/ www.exalitica.com http://exalitica.com/web/img/frim.png

S3 table to spark sql

2014-11-11 Thread Franco Barrientos
problema i show can i transform string variable into date variables (fechau3m)? Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 mailto:franco.barrien...@exalitica.com franco.barrien...@exalitica.com http://www.exalitica.com