in Hive 2, I don't see this issue INSERT/SELECT from INT to String column!
Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 19 July 2016 at 20:39, Mahender Sarangam <mahender.bigd...@outlook.com> wrote: > > Thanks Matthew, > > Currently we are in Hive 1.2 version only, Is there any setting like > "hive.metastore.disallow.incompatible.col.type.changes=false;" in Hive 1.2 > or any around apart for reloading entire table data. For Quick workaround, > we are reloading entire data. > Can you please share with us Jira for Schema Evolution. > > > @Mich : Currently we have only primitive types. But I'm also interested to > know "how the behavior will be in complex types" > > > /Mahender > > > On 7/18/2016 3:55 PM, Mich Talebzadeh wrote: > > Hi Mathew, > > In layman's term if I create the source ORC table column as INT and then > create a target ORC table but that column has now been defined as STRING > and do an INSERT/SELECT from source table how data is internally stored? > > Is it implicitly converted into new format using CAST function or it is > stored as is and just masked? > > The version of Hive I am using is 2 and it works OK for primitive data > types (insert/select from INT to String) > > However, I believe Mahender is referring to Complex types? > > Thanks > > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 18 July 2016 at 22:31, Matthew McCline <mmccl...@hortonworks.com> > wrote: > >> >> Hi Mahender, >> >> >> Schema Evolution is available on the latest recent version of Hive. >> >> >> For example, if you set >> hive.metastore.disallow.incompatible.col.type.changes=false; on master >> (i.e. hive2) it will support INT to STRING conversion. >> >> >> If you need to remain on an older version, then you are out of luck. >> >> >> Thanks, >> >> Matt >> >> >> ------------------------------ >> *From:* Mahender Sarangam <mahender.bigd...@outlook.com> >> *Sent:* Monday, July 18, 2016 1:59 PM >> *To:* user@hive.apache.org >> *Subject:* Re: ORC does not support type conversion from INT to STRING. >> >> >> Hi Mich, >> >> Sorry for delay in responding. here is the scenario, >> >> We have created new cluster and we have moved all ORC File data into new >> cluster. We have re-created table pointing to ORC location. We have >> modified data type of ORC table from *INT *to *String.* From then >> onward, we were unable to fire select statement against this ORC table, >> hive keep throwing exception, "Orc table select. Unable to convert Int to >> String". Looks like it is bug in ORC table only. Where in we modify the >> datatype from *int to string,* is causing problem with ORC >> reading/select statement, it throws exceptio. Please let me know if there >> are any workaround for this scenario. Is this behavior expected previously >> also. >> >> >> */Mahender* >> >> >> >> >> >> >> On 6/14/2016 11:47 AM, Mich Talebzadeh wrote: >> >> you must excuse my ignorance >> >> can you please elaborate on this as there seems something has gone wrong >> somewhere? >> >> Dr Mich Talebzadeh >> >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> >> On 14 June 2016 at 19:42, Mahender Sarangam <mahender.bigd...@outlook.com >> > wrote: >> >>> Yes Mich. We have restored cluster from metastore. >>> >>> On 6/14/2016 11:35 AM, Mich Talebzadeh wrote: >>> >>> Hi Mahendar, >>> >>> >>> Did you load the meta-data DB/schema from backup and now seeing this >>> error >>> >>> >>> >>> Dr Mich Talebzadeh >>> >>> >>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> >>> http://talebzadehmich.wordpress.com >>> >>> >>> >>> On 14 June 2016 at 19:04, Mahender Sarangam < >>> mahender.bigd...@outlook.com> wrote: >>> >>>> ping. >>>> >>>> On 6/13/2016 1:19 PM, Mahender Sarangam wrote: >>>> >>>> Hi, >>>> >>>> We are facing issue while reading data from ORC table. We have created >>>> ORC table and dumped data into it. We have deleted cluster due to some >>>> reason. When we recreated cluster (using Metastore) and table pointing to >>>> same location. When we perform reading from ORC table. We see below error. >>>> >>>> SELECT col2, Col1, >>>> reflect("java.util.UUID", "randomUUID") AS ID, >>>> Source, >>>> 1 , >>>> SDate, >>>> EDate >>>> FROM Table ORC JOIN Table2 _surr; >>>> >>>> ERROR : Vertex failed, vertexName=Map 1, >>>> vertexId=vertex_1465411930667_0212_1_01, diagnostics=[Task failed, >>>> taskId=task_1465411930667_0212_1_01_000000, diagnostics=[TaskAttempt 0 >>>> failed, info=[Error: Failure while running task:java.lang.RuntimeException: >>>> java.lang.RuntimeException: java.io.IOException: java.io.IOException: ORC >>>> does not support type conversion from INT to STRING. >>>> >>>> >>>> I think issue is reflect("java.util.UUID", "randomUUID") AS ID >>>> >>>> >>>> I know there is Bug raised while reading data from ORC table. Is there >>>> any workaround apart from reloading data. >>>> >>>> -MS >>>> >>>> >>>> >>>> >>>> >>> >>> >> >> > >
