can't get JDBC type for array

[Suresh Ayyavoo]

Save below dataframe to database cause error "can't get JDBC type for
array"


root

 |-- accountID: array (nullable = true)

 ||-- element: string (containsNull = true)

 |-- completed_at: array (nullable = true)

 ||-- element: string (containsNull = true)

 |-- created_at: array (nullable = true)

 ||-- element: string (containsNull = true)


Exception in thread "main" java.lang.IllegalArgumentException: Can't get
JDBC type for array


Thanks

Re: Release on 0.7.1 and 0.7.2

[Jun Kim]

Cool! I look forward to it!

2017년 3월 14일 (화) 오후 12:31, moon soo Lee 님이 작성:

> Sounds like a plan!
>
>
> On Mon, Mar 13, 2017 at 8:22 PM Xiaohui Liu  wrote:
>
> This is the right action. In fact, 0.7.0 release bin did not work for my
> team. We almost started to use 0.7.1-snapshot immediately after 0.7.0
> release.
>
> I guess many of us are taking the same route.
>
> But for new zeppelin users, starting with 0.7.0 will give them the wrong
> first impression.
>
>
> On Tue, 14 Mar 2017 at 10:28 AM, Jongyoul Lee  wrote:
>
> Hi dev and users,
>
> As we released 0.7.0, most of users and dev reported a lot of bugs which
> were critical. For the reason, community including me started to prepare
> new minor release with umbrella issue[1]. Due to contributors' efforts, we
> have resolved some of issues and have reviewed almost unresolved issues. I
> want to talk about the new minor release at this point. Generally, we have
> resolved all of issues reported as bugs before we release but some issues
> are very critical and it causes serious problem using Apache Zeppelin. Then
> I think, in this time, it's better to release 0.7.1 as soon as we can and
> prepare a new minor release with rest of unresolved issues.
>
> I'd like to start a process this Friday and if some issues are not merged
> until then, I hope they would be included in 0.7.2.
>
> Feel free to talk to me if you have a better plan to improve users'
> experiences.
>
> Regards,
> Jongyoul Lee
>
> [1] https://issues.apache.org/jira/browse/ZEPPELIN-2134
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
> --
Taejun Kim

Data Mining Lab.
School of Electrical and Computer Engineering
University of Seoul

Re: Release on 0.7.1 and 0.7.2

[moon soo Lee]

Sounds like a plan!

On Mon, Mar 13, 2017 at 8:22 PM Xiaohui Liu  wrote:

> This is the right action. In fact, 0.7.0 release bin did not work for my
> team. We almost started to use 0.7.1-snapshot immediately after 0.7.0
> release.
>
> I guess many of us are taking the same route.
>
> But for new zeppelin users, starting with 0.7.0 will give them the wrong
> first impression.
>
>
> On Tue, 14 Mar 2017 at 10:28 AM, Jongyoul Lee  wrote:
>
> Hi dev and users,
>
> As we released 0.7.0, most of users and dev reported a lot of bugs which
> were critical. For the reason, community including me started to prepare
> new minor release with umbrella issue[1]. Due to contributors' efforts, we
> have resolved some of issues and have reviewed almost unresolved issues. I
> want to talk about the new minor release at this point. Generally, we have
> resolved all of issues reported as bugs before we release but some issues
> are very critical and it causes serious problem using Apache Zeppelin. Then
> I think, in this time, it's better to release 0.7.1 as soon as we can and
> prepare a new minor release with rest of unresolved issues.
>
> I'd like to start a process this Friday and if some issues are not merged
> until then, I hope they would be included in 0.7.2.
>
> Feel free to talk to me if you have a better plan to improve users'
> experiences.
>
> Regards,
> Jongyoul Lee
>
> [1] https://issues.apache.org/jira/browse/ZEPPELIN-2134
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>

Re: Release on 0.7.1 and 0.7.2

[Xiaohui Liu]

This is the right action. In fact, 0.7.0 release bin did not work for my
team. We almost started to use 0.7.1-snapshot immediately after 0.7.0
release.

I guess many of us are taking the same route.

But for new zeppelin users, starting with 0.7.0 will give them the wrong
first impression.


On Tue, 14 Mar 2017 at 10:28 AM, Jongyoul Lee  wrote:

> Hi dev and users,
>
> As we released 0.7.0, most of users and dev reported a lot of bugs which
> were critical. For the reason, community including me started to prepare
> new minor release with umbrella issue[1]. Due to contributors' efforts, we
> have resolved some of issues and have reviewed almost unresolved issues. I
> want to talk about the new minor release at this point. Generally, we have
> resolved all of issues reported as bugs before we release but some issues
> are very critical and it causes serious problem using Apache Zeppelin. Then
> I think, in this time, it's better to release 0.7.1 as soon as we can and
> prepare a new minor release with rest of unresolved issues.
>
> I'd like to start a process this Friday and if some issues are not merged
> until then, I hope they would be included in 0.7.2.
>
> Feel free to talk to me if you have a better plan to improve users'
> experiences.
>
> Regards,
> Jongyoul Lee
>
> [1] https://issues.apache.org/jira/browse/ZEPPELIN-2134
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: How can i convert below DataFrame array to string

[Suresh Ayyavoo]

Thanks also below code helped

df.select(concat_ws(",",$"result.accountID") as "accountID")

On Tue, Mar 14, 2017 at 5:35 AM, Felix Cheung 
wrote:

> Explode?
>
> https://spark.apache.org/docs/latest/api/python/pyspark.sql.
> html?highlight=explode#pyspark.sql.functions.explode
>
>
> For Spark questions like this, try stackoverflow
>
> spark.apache.org/community.html
>
>
>
>
> --
> *From:* Suresh Ayyavoo 
> *Sent:* Sunday, March 12, 2017 8:53:52 PM
> *To:* users@zeppelin.apache.org
> *Subject:* How can i convert below DataFrame array to string
>
> Hi All,
>
> How can i convert below DataFrame [accountID: array,
> completed_at: array, created_at: array] to
> [accountID: string, completed_at: string, created_at: string]
>
> root
>
>  |-- accountID: array (nullable = true)
>
>  ||-- element: string (containsNull = true)
>
>  |-- completed_at: array (nullable = true)
>
>  ||-- element: string (containsNull = true)
>
>  |-- created_at: array (nullable = true)
>
>  ||-- element: string (containsNull = true)
>
>
> root
> |-- accountID: string (containsNull =true)
> |-- completed_at: string (containsNull =true)
> |--create_at : string (containsNull =true)
>
>
> Thanks
>
>
>


-- 

Suresh Ayyavoo

Solution Architect / R Lead

iAPPS Pte. Ltd.

3 Fusionopolis Way, Symbiosis #13-25 S(138633)

[O] 64631795   [F] 6778 5300 [M] 91540224

Website: www.iappsasia.com

Facebook: www.facebook.com/iappsasia

Youtube: www.youtube.com/user/iAPPSasia

Release on 0.7.1 and 0.7.2

[Jongyoul Lee]

Hi dev and users,

As we released 0.7.0, most of users and dev reported a lot of bugs which
were critical. For the reason, community including me started to prepare
new minor release with umbrella issue[1]. Due to contributors' efforts, we
have resolved some of issues and have reviewed almost unresolved issues. I
want to talk about the new minor release at this point. Generally, we have
resolved all of issues reported as bugs before we release but some issues
are very critical and it causes serious problem using Apache Zeppelin. Then
I think, in this time, it's better to release 0.7.1 as soon as we can and
prepare a new minor release with rest of unresolved issues.

I'd like to start a process this Friday and if some issues are not merged
until then, I hope they would be included in 0.7.2.

Feel free to talk to me if you have a better plan to improve users'
experiences.

Regards,
Jongyoul Lee

[1] https://issues.apache.org/jira/browse/ZEPPELIN-2134

-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: How can i convert below DataFrame array to string

[Felix Cheung]

Explode?

https://spark.apache.org/docs/latest/api/python/pyspark.sql.html?highlight=explode#pyspark.sql.functions.explode


For Spark questions like this, try stackoverflow

spark.apache.org/community.html




From: Suresh Ayyavoo 
Sent: Sunday, March 12, 2017 8:53:52 PM
To: users@zeppelin.apache.org
Subject: How can i convert below DataFrame array to string

Hi All,

How can i convert below DataFrame [accountID: array, completed_at: 
array, created_at: array] to
[accountID: string, completed_at: string, created_at: string]


root

 |-- accountID: array (nullable = true)

 ||-- element: string (containsNull = true)

 |-- completed_at: array (nullable = true)

 ||-- element: string (containsNull = true)

 |-- created_at: array (nullable = true)

 ||-- element: string (containsNull = true)


root
|-- accountID: string (containsNull =true)
|-- completed_at: string (containsNull =true)
|--create_at : string (containsNull =true)


Thanks

Install Zeppelin on Windows to connect to remote Hadoop cluster

[Abhi Basu]

Can you please point me to documentation to set up an install of Zeppelin
on Windows to connect to remote Hadoop cluster?

Thanks,

Abhi

-- 
Abhi Basu

Re: "spark ui" button in spark interpreter does not show Spark web-ui

[Hyung Sung Shim]

Hello.
Thank you for sharing the problem.
Could you file a jira issue for this?

2017년 3월 13일 (월) 오후 3:18, Meethu Mathew 님이 작성:

> Hi,
>
> I have noticed the same problem
>
> Regards,
>
>
> Meethu Mathew
>
>
> On Mon, Mar 13, 2017 at 9:56 AM, Xiaohui Liu  wrote:
>
> Hi,
>
> We used 0.7.1-snapshot with our Mesos cluster, almost all our needed
> features (ldap login, notebook acl control, livy/pyspark/rspark/scala,
> etc.) work pretty well.
>
> But one thing does not work for us is the 'spark ui' button does not
> response to user clicks. No errors in browser side.
>
> Anyone has met similar issues? Any suggestions about where I should check?
>
> Regards
> Xiaohui
>
>
>

Re: "spark ui" button in spark interpreter does not show Spark web-ui

[Meethu Mathew]

Hi,

I have noticed the same problem

Regards,
Meethu Mathew


On Mon, Mar 13, 2017 at 9:56 AM, Xiaohui Liu  wrote:

> Hi,
>
> We used 0.7.1-snapshot with our Mesos cluster, almost all our needed
> features (ldap login, notebook acl control, livy/pyspark/rspark/scala,
> etc.) work pretty well.
>
> But one thing does not work for us is the 'spark ui' button does not
> response to user clicks. No errors in browser side.
>
> Anyone has met similar issues? Any suggestions about where I should check?
>
> Regards
> Xiaohui
>