RE: DIH serialize
Dear Stefan,
thank you for your help!
Well, I wrote a small script, even if not json, but works:
<![CDATA[
function my_serialize(row)
{
var st = "";
st = row.get('stt_id') + "||" +
row.get('stt_name') + "||" +
row.get('stt_date_from') + "||" +
row.get('stt_date_to') + "||" +
row.get('stt_monday') + "||" +
row.get('stt_tuesday') + "||" +
row.get('stt_wednesday') + "||" +
row.get('stt_thursday') + "||" +
row.get('stt_friday') + "||" +
row.get('stt_saturday') + "||" +
row.get('stt_sunday') ;
var ret = new java.util.HashMap();
ret.put('main_timetable', st);
return ret;
}
]]>
regards,
Rich
-Original Message-
From: Stefan Matheis [mailto:matheis.ste...@googlemail.com]
Sent: Tuesday, January 25, 2011 11:13
To: solr-user@lucene.apache.org
Subject: Re: DIH serialize
Rich,
i played around for a few minutes with Script-Transformers, but i have not
enough knowledge to get anything done right know :/
My Idea was: looping over the given row, which should be a Java HashMap or
something like that? and do sth like this (pseudo-code):
var row_data = []
for( var key in row )
{
row_data.push( '"' + key + '" : '" + row[key] + '"' );
}
row.put( 'whatever_field', '{' + row_data.join( ',' ) + '}' );
Which should result in a json-object like {'key1':'value1', 'key2':'value2'}
- and that should be okay to work with?
Regards
Stefan
On Mon, Jan 24, 2011 at 7:53 PM, Papp Richard wrote:
> Hi Stefan,
>
> yes, this is exactly what I intend - I don't want to search in this field
> - just quicly return me the result in a serialized form (the search
> criteria
> is on other fields). Well, if I could serialize the data exactly as like
> the
> PHP serialize() does I would be maximally satisfied, but any other form in
> which I could compact the data easily into one field I would be pleased.
> Can anyone help me? I guess the is quite a good way, but I don't
> know which function should I use there to compact the data to be easily
> usable in PHP. Or any other method?
>
> thanks,
> Rich
>
> -Original Message-
> From: Stefan Matheis [<a rel="nofollow" href="mailto:matheis.ste...@googlemail.com">mailto:matheis.ste...@googlemail.com</a>]
> Sent: Monday, January 24, 2011 18:23
> To: solr-user@lucene.apache.org
> Subject: Re: DIH serialize
>
> Hi Rich,
>
> i'm a bit confused after reading your post .. what exactly you wanna try
to
> achieve? Serializing (like <a rel="nofollow" href="http://php.net/serialize">http://php.net/serialize</a>) your complete row
> into
> one field? Don't wanna search in them, just store and deliver them in your
> results? Does that make sense? Sounds a bit strange :)
>
> Regards
> Stefan
>
> On Mon, Jan 24, 2011 at 10:03 AM, Papp Richard <ccode...@gmail.com> wrote:
>
> > Hi Dennis,
> >
> > thank you for your answer, but didn't understand why you say it doesn't
> > need serialization. I'm with the option "C".
> > but the main question is, how to put into one field a result of many
> > fields: "SELECT * FROM".
> >
> > thanks,
> > Rich
> >
> > -Original Message-
> > From: Dennis Gearon [<a rel="nofollow" href="mailto:gear...@sbcglobal.net">mailto:gear...@sbcglobal.net</a>]
> > Sent: Monday, January 24, 2011 02:07
> > To: solr-user@lucene.apache.org
> > Subject: Re: DIH serialize
> >
> > Depends on your process chain to the eventual viewer/consumer of the
> data.
> >
> > The questions to ask are:
> > A/ Is the data IN Solr going to be viewed or processed in its original
> > form:
> > -->set stored = 'true'
> > --->no serialization needed.
> > B/ If it's going to be anayzed and searched for separate from any other
> > field,
> >
> > the analyzing will put it into an unreadable form. If you need to
> see
> > it,
> > then
> > --->set indexed="true" and stored="true"
> > --->no serializaton needed. C/ If it's NOT going to be viewed AS
> IS,
> > and
> > it's not going to be searched for AS IS,
&
Re: DIH serialize
Rich,
i played around for a few minutes with Script-Transformers, but i have not
enough knowledge to get anything done right know :/
My Idea was: looping over the given row, which should be a Java HashMap or
something like that? and do sth like this (pseudo-code):
var row_data = []
for( var key in row )
{
row_data.push( '"' + key + '" : '" + row[key] + '"' );
}
row.put( 'whatever_field', '{' + row_data.join( ',' ) + '}' );
Which should result in a json-object like {'key1':'value1', 'key2':'value2'}
- and that should be okay to work with?
Regards
Stefan
On Mon, Jan 24, 2011 at 7:53 PM, Papp Richard wrote:
> Hi Stefan,
>
> yes, this is exactly what I intend - I don't want to search in this field
> - just quicly return me the result in a serialized form (the search
> criteria
> is on other fields). Well, if I could serialize the data exactly as like
> the
> PHP serialize() does I would be maximally satisfied, but any other form in
> which I could compact the data easily into one field I would be pleased.
> Can anyone help me? I guess the is quite a good way, but I don't
> know which function should I use there to compact the data to be easily
> usable in PHP. Or any other method?
>
> thanks,
> Rich
>
> -Original Message-
> From: Stefan Matheis [<a rel="nofollow" href="mailto:matheis.ste...@googlemail.com">mailto:matheis.ste...@googlemail.com</a>]
> Sent: Monday, January 24, 2011 18:23
> To: solr-user@lucene.apache.org
> Subject: Re: DIH serialize
>
> Hi Rich,
>
> i'm a bit confused after reading your post .. what exactly you wanna try to
> achieve? Serializing (like <a rel="nofollow" href="http://php.net/serialize">http://php.net/serialize</a>) your complete row
> into
> one field? Don't wanna search in them, just store and deliver them in your
> results? Does that make sense? Sounds a bit strange :)
>
> Regards
> Stefan
>
> On Mon, Jan 24, 2011 at 10:03 AM, Papp Richard <ccode...@gmail.com> wrote:
>
> > Hi Dennis,
> >
> > thank you for your answer, but didn't understand why you say it doesn't
> > need serialization. I'm with the option "C".
> > but the main question is, how to put into one field a result of many
> > fields: "SELECT * FROM".
> >
> > thanks,
> > Rich
> >
> > -Original Message-
> > From: Dennis Gearon [<a rel="nofollow" href="mailto:gear...@sbcglobal.net">mailto:gear...@sbcglobal.net</a>]
> > Sent: Monday, January 24, 2011 02:07
> > To: solr-user@lucene.apache.org
> > Subject: Re: DIH serialize
> >
> > Depends on your process chain to the eventual viewer/consumer of the
> data.
> >
> > The questions to ask are:
> > A/ Is the data IN Solr going to be viewed or processed in its original
> > form:
> > -->set stored = 'true'
> > --->no serialization needed.
> > B/ If it's going to be anayzed and searched for separate from any other
> > field,
> >
> > the analyzing will put it into an unreadable form. If you need to
> see
> > it,
> > then
> > --->set indexed="true" and stored="true"
> > --->no serializaton needed. C/ If it's NOT going to be viewed AS
> IS,
> > and
> > it's not going to be searched for AS IS,
> > (i.e. other columns will be how the data is found), and you have
> > another,
> >
> > serialzable format:
> > -->set indexed="false" and stored="true"
> > -->serialize AS PER THE INTENDED APPLICATION,
> > not sure that Solr can do that at all.
> > C/ If it's NOT going to be viewed AS IS, and it's not going to be
> searched
> > for
> > AS IS,
> > (i.e. other columns will be how the data is found), and you have
> > another,
> >
> > serialzable format:
> > -->set indexed="false" and stored="true"
> > -->serialize AS PER THE INTENDED APPLICATION,
> > not sure that Solr can do that at all.
> > D/ If it's NOT going to be viewed AS IS, BUT it's going to be searched
> for
> > AS
> > IS,
> > (this column will be how the data is found), and you have another,
> > serialzable format:
> > -->you need to put it into TWO columns
> > -->A SERIALIZED FIELD
> > -->set index
Re: DIH serialize
UNSUBSCRIBE
On 1/23/11, Papp Richard wrote:
> Hi all,
>
>
>
> I wasted the last few hours trying to serialize some column values (from
> mysql) into a Solr column, but I just can't find such a function. I'll use
> the value in PHP - I don't know if it is possible to serialize in PHP style
> at all. This is what I tried and works with a given factor:
>
>
>
> in schema.xml:
>
> stored="true" multiValued="true" />
>
>
>
> in DIH xml:
>
>
>
>
>
>
> function my_serialize(row)
>
> {
>
> row.put('main_timetable', row.toString());
>
> return row;
>
> }
>
> ]]>
>
>
>
> .
>
>
>
>
> transformer="script:my_serialize"
>
> >
>
> .
>
>>
>
>
>
> Can I use java directly in script (
RE: DIH serialize
Hi Stefan,
yes, this is exactly what I intend - I don't want to search in this field
- just quicly return me the result in a serialized form (the search criteria
is on other fields). Well, if I could serialize the data exactly as like the
PHP serialize() does I would be maximally satisfied, but any other form in
which I could compact the data easily into one field I would be pleased.
Can anyone help me? I guess the is quite a good way, but I don't
know which function should I use there to compact the data to be easily
usable in PHP. Or any other method?
thanks,
Rich
-Original Message-
From: Stefan Matheis [<a rel="nofollow" href="mailto:matheis.ste...@googlemail.com">mailto:matheis.ste...@googlemail.com</a>]
Sent: Monday, January 24, 2011 18:23
To: solr-user@lucene.apache.org
Subject: Re: DIH serialize
Hi Rich,
i'm a bit confused after reading your post .. what exactly you wanna try to
achieve? Serializing (like <a rel="nofollow" href="http://php.net/serialize">http://php.net/serialize</a>) your complete row into
one field? Don't wanna search in them, just store and deliver them in your
results? Does that make sense? Sounds a bit strange :)
Regards
Stefan
On Mon, Jan 24, 2011 at 10:03 AM, Papp Richard <ccode...@gmail.com> wrote:
> Hi Dennis,
>
> thank you for your answer, but didn't understand why you say it doesn't
> need serialization. I'm with the option "C".
> but the main question is, how to put into one field a result of many
> fields: "SELECT * FROM".
>
> thanks,
> Rich
>
> -Original Message-
> From: Dennis Gearon [<a rel="nofollow" href="mailto:gear...@sbcglobal.net">mailto:gear...@sbcglobal.net</a>]
> Sent: Monday, January 24, 2011 02:07
> To: solr-user@lucene.apache.org
> Subject: Re: DIH serialize
>
> Depends on your process chain to the eventual viewer/consumer of the data.
>
> The questions to ask are:
> A/ Is the data IN Solr going to be viewed or processed in its original
> form:
> -->set stored = 'true'
> --->no serialization needed.
> B/ If it's going to be anayzed and searched for separate from any other
> field,
>
> the analyzing will put it into an unreadable form. If you need to
see
> it,
> then
> --->set indexed="true" and stored="true"
> --->no serializaton needed. C/ If it's NOT going to be viewed AS IS,
> and
> it's not going to be searched for AS IS,
> (i.e. other columns will be how the data is found), and you have
> another,
>
> serialzable format:
> -->set indexed="false" and stored="true"
> -->serialize AS PER THE INTENDED APPLICATION,
> not sure that Solr can do that at all.
> C/ If it's NOT going to be viewed AS IS, and it's not going to be
searched
> for
> AS IS,
> (i.e. other columns will be how the data is found), and you have
> another,
>
> serialzable format:
> -->set indexed="false" and stored="true"
> -->serialize AS PER THE INTENDED APPLICATION,
> not sure that Solr can do that at all.
> D/ If it's NOT going to be viewed AS IS, BUT it's going to be searched
for
> AS
> IS,
> (this column will be how the data is found), and you have another,
> serialzable format:
> -->you need to put it into TWO columns
> -->A SERIALIZED FIELD
> -->set indexed="false" and stored="true"
>
> -->>AN UNSERIALIZED FIELD
> -->set indexed="false" and stored="true"
> -->serialize AS PER THE INTENDED APPLICATION,
> not sure that Solr can do that at all.
>
> Hope that helps!
>
>
> Dennis Gearon
>
>
> Signature Warning
> ----
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others' mistakes, so you do not have to make them
> yourself.
> from '<a rel="nofollow" href="http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036">http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036</a>'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
>
> - Original Message
> From: Papp Richard <ccode...@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Sun, January 23, 2011 2:02:05 PM
> Subject: DIH serialize
>
> Hi all,
>
>
>
> I wasted the last few hours trying to serialize some column values (from
> mysql) into a Solr column, but I just can't find such a function. I'll use
> the value in PHP - I don't know if it is possible to serialize in PHP
style
> at all. This is what I tried and works with a given factor:
>
>
>
> in schema.xml:
>
> <field name="main_timetable" type="text" indexed="false"
> stored="true" multiValued="true" />
>
>
>
> in DIH xml:
>
>
>
> <dataConfig>
>
> <script><![CDATA[
>
>function my_serialize(row)
>
>{
>
> row.put('main_timetable', row.toString());
>
> return row;
>
>}
>
> ]]>
>
>
>
> .
>
>
>
>
>transformer="script:my_serialize"
>
>>
>
> .
>
> >
>
>
>
> Can I use java directly in script (
Re: DIH serialize
Hi Rich,
i'm a bit confused after reading your post .. what exactly you wanna try to
achieve? Serializing (like http://php.net/serialize) your complete row into
one field? Don't wanna search in them, just store and deliver them in your
results? Does that make sense? Sounds a bit strange :)
Regards
Stefan
On Mon, Jan 24, 2011 at 10:03 AM, Papp Richard wrote:
> Hi Dennis,
>
> thank you for your answer, but didn't understand why you say it doesn't
> need serialization. I'm with the option "C".
> but the main question is, how to put into one field a result of many
> fields: "SELECT * FROM".
>
> thanks,
> Rich
>
> -Original Message-
> From: Dennis Gearon [mailto:gear...@sbcglobal.net]
> Sent: Monday, January 24, 2011 02:07
> To: solr-user@lucene.apache.org
> Subject: Re: DIH serialize
>
> Depends on your process chain to the eventual viewer/consumer of the data.
>
> The questions to ask are:
> A/ Is the data IN Solr going to be viewed or processed in its original
> form:
> -->set stored = 'true'
> --->no serialization needed.
> B/ If it's going to be anayzed and searched for separate from any other
> field,
>
> the analyzing will put it into an unreadable form. If you need to see
> it,
> then
> --->set indexed="true" and stored="true"
> --->no serializaton needed. C/ If it's NOT going to be viewed AS IS,
> and
> it's not going to be searched for AS IS,
> (i.e. other columns will be how the data is found), and you have
> another,
>
> serialzable format:
> -->set indexed="false" and stored="true"
> -->serialize AS PER THE INTENDED APPLICATION,
> not sure that Solr can do that at all.
> C/ If it's NOT going to be viewed AS IS, and it's not going to be searched
> for
> AS IS,
> (i.e. other columns will be how the data is found), and you have
> another,
>
> serialzable format:
> -->set indexed="false" and stored="true"
> -->serialize AS PER THE INTENDED APPLICATION,
> not sure that Solr can do that at all.
> D/ If it's NOT going to be viewed AS IS, BUT it's going to be searched for
> AS
> IS,
> (this column will be how the data is found), and you have another,
> serialzable format:
> -->you need to put it into TWO columns
> -->A SERIALIZED FIELD
> -->set indexed="false" and stored="true"
>
> -->>AN UNSERIALIZED FIELD
> -->set indexed="false" and stored="true"
> -->serialize AS PER THE INTENDED APPLICATION,
> not sure that Solr can do that at all.
>
> Hope that helps!
>
>
> Dennis Gearon
>
>
> Signature Warning
> --------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
>
> - Original Message
> From: Papp Richard
> To: solr-user@lucene.apache.org
> Sent: Sun, January 23, 2011 2:02:05 PM
> Subject: DIH serialize
>
> Hi all,
>
>
>
> I wasted the last few hours trying to serialize some column values (from
> mysql) into a Solr column, but I just can't find such a function. I'll use
> the value in PHP - I don't know if it is possible to serialize in PHP style
> at all. This is what I tried and works with a given factor:
>
>
>
> in schema.xml:
>
>stored="true" multiValued="true" />
>
>
>
> in DIH xml:
>
>
>
>
>
> <![CDATA[
>
>function my_serialize(row)
>
>{
>
> row.put('main_timetable', row.toString());
>
> return row;
>
>}
>
> ]]>
>
>
>
> .
>
>
>
>
>transformer="script:my_serialize"
>
>>
>
> .
>
> >
>
>
>
> Can I use java directly in script (
RE: DIH serialize
Hi Dennis,
thank you for your answer, but didn't understand why you say it doesn't need
serialization. I'm with the option "C".
but the main question is, how to put into one field a result of many fields:
"SELECT * FROM".
thanks,
Rich
-Original Message-
From: Dennis Gearon [mailto:gear...@sbcglobal.net]
Sent: Monday, January 24, 2011 02:07
To: solr-user@lucene.apache.org
Subject: Re: DIH serialize
Depends on your process chain to the eventual viewer/consumer of the data.
The questions to ask are:
A/ Is the data IN Solr going to be viewed or processed in its original form:
-->set stored = 'true'
--->no serialization needed.
B/ If it's going to be anayzed and searched for separate from any other
field,
the analyzing will put it into an unreadable form. If you need to see
it,
then
--->set indexed="true" and stored="true"
--->no serializaton needed. C/ If it's NOT going to be viewed AS IS, and
it's not going to be searched for AS IS,
(i.e. other columns will be how the data is found), and you have
another,
serialzable format:
-->set indexed="false" and stored="true"
-->serialize AS PER THE INTENDED APPLICATION,
not sure that Solr can do that at all.
C/ If it's NOT going to be viewed AS IS, and it's not going to be searched
for
AS IS,
(i.e. other columns will be how the data is found), and you have
another,
serialzable format:
-->set indexed="false" and stored="true"
-->serialize AS PER THE INTENDED APPLICATION,
not sure that Solr can do that at all.
D/ If it's NOT going to be viewed AS IS, BUT it's going to be searched for AS
IS,
(this column will be how the data is found), and you have another,
serialzable format:
-->you need to put it into TWO columns
-->A SERIALIZED FIELD
-->set indexed="false" and stored="true"
-->>AN UNSERIALIZED FIELD
-->set indexed="false" and stored="true"
-->serialize AS PER THE INTENDED APPLICATION,
not sure that Solr can do that at all.
Hope that helps!
Dennis Gearon
Signature Warning
It is always a good idea to learn from your own mistakes. It is usually a
better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
EARTH has a Right To Life,
otherwise we all die.
- Original Message
From: Papp Richard
To: solr-user@lucene.apache.org
Sent: Sun, January 23, 2011 2:02:05 PM
Subject: DIH serialize
Hi all,
I wasted the last few hours trying to serialize some column values (from
mysql) into a Solr column, but I just can't find such a function. I'll use
the value in PHP - I don't know if it is possible to serialize in PHP style
at all. This is what I tried and works with a given factor:
in schema.xml:
in DIH xml:
<![CDATA[
function my_serialize(row)
{
row.put('main_timetable', row.toString());
return row;
}
]]>
.
.
>
Can I use java directly in script (
Re: DIH serialize
Depends on your process chain to the eventual viewer/consumer of the data.
The questions to ask are:
A/ Is the data IN Solr going to be viewed or processed in its original form:
-->set stored = 'true'
--->no serialization needed.
B/ If it's going to be anayzed and searched for separate from any other
field,
the analyzing will put it into an unreadable form. If you need to see
it,
then
--->set indexed="true" and stored="true"
--->no serializaton needed. C/ If it's NOT going to be viewed AS IS, and
it's not going to be searched for AS IS,
(i.e. other columns will be how the data is found), and you have
another,
serialzable format:
-->set indexed="false" and stored="true"
-->serialize AS PER THE INTENDED APPLICATION,
not sure that Solr can do that at all.
C/ If it's NOT going to be viewed AS IS, and it's not going to be searched
for
AS IS,
(i.e. other columns will be how the data is found), and you have
another,
serialzable format:
-->set indexed="false" and stored="true"
-->serialize AS PER THE INTENDED APPLICATION,
not sure that Solr can do that at all.
D/ If it's NOT going to be viewed AS IS, BUT it's going to be searched for AS
IS,
(this column will be how the data is found), and you have another,
serialzable format:
-->you need to put it into TWO columns
-->A SERIALIZED FIELD
-->set indexed="false" and stored="true"
-->>AN UNSERIALIZED FIELD
-->set indexed="false" and stored="true"
-->serialize AS PER THE INTENDED APPLICATION,
not sure that Solr can do that at all.
Hope that helps!
Dennis Gearon
Signature Warning
It is always a good idea to learn from your own mistakes. It is usually a
better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
EARTH has a Right To Life,
otherwise we all die.
- Original Message
From: Papp Richard
To: solr-user@lucene.apache.org
Sent: Sun, January 23, 2011 2:02:05 PM
Subject: DIH serialize
Hi all,
I wasted the last few hours trying to serialize some column values (from
mysql) into a Solr column, but I just can't find such a function. I'll use
the value in PHP - I don't know if it is possible to serialize in PHP style
at all. This is what I tried and works with a given factor:
in schema.xml:
in DIH xml:
<![CDATA[
function my_serialize(row)
{
row.put('main_timetable', row.toString());
return row;
}
]]>
.
.
>
Can I use java directly in script (
DIH serialize
Hi all, I wasted the last few hours trying to serialize some column values (from mysql) into a Solr column, but I just can't find such a function. I'll use the value in PHP - I don't know if it is possible to serialize in PHP style at all. This is what I tried and works with a given factor: in schema.xml: in DIH xml: . . > Can I use java directly in script (
