date:20180223

Re: data import - product images

2018-02-23 Thread Rishi Solanki

Jesse,
Assuming that you have a wrapper service which call resize images and data
store service for each product. Lets say wrapperService() is your main
service and inlineProductUpdateSerive() is the service which invoke in each
iteration in wrapperService(). The inlineProductUpdateSerive() cakk in the
wrapperService should look like this;

dispatcher.runSync("inlineProductUpdateSerive", 300, true, serviceInCtx);

OR

dispatcher.runAsync("inlineProductUpdateSerive", true, serviceInCtx);

In first option above, if service will return error then simply log it so
that service will continue run for other products. In second case true
value passed for persist that means it will re try 3 times on failure.

Bottom line is no need to handle the transaction manually OFBiz service
engine facilitate you to do that by just passing some flags. Finally
increase the transaction time in the service definition of wrapperService()
and call the inlineProductUpdateSerive() as mentioned above.

HTH!

--

Rishi Solanki
Sr Manager, Enterprise Software Development
HotWax Systems Pvt. Ltd.
Direct: +91-9893287847
http://www.hotwaxsystems.com
www.hotwax.co

On Fri, Feb 23, 2018 at 2:25 PM, gil portenseigne <
gil.portensei...@nereide.fr> wrote:

> Hello Jesse,
>
> You could manually manage your transactions within your service using :
>
> actualTransaction = TransactionUtil.suspend(); //suspend the current
> transaction
> TransactionUtil.begin(600); //Start a new one
> [...]
> if (ServiceUtil.isError(result)) {
> TransactionUtil.rollback(); //Rollback the new one
> } else {
> TransactionUtil.commit(); // Or Commit it
> }
> TransactionUtil.resume(actualTransaction); //resume the suspended
> transaction.
>
> With this solution you keep your process sync, and you can manage errors.
>
> HTH
>
> Gil
>
>
>
> On 23/02/2018 09:29, Jesse Thomas wrote:
>
>> I have written a service that checks a folder for image files, resizes
>> and updates the Product in OFBiz based on the file name equaling the
>> productId. It works for a few hundred images, but when I process a few
>> thousand images at the end it hits a transaction timeout. All the images
>> are resized and saved. But the product records don't get updated. All I'm
>> doing in the database is updating the Product image fields, is there a way
>> to force the commit on each product instead at the end of the service call?
>>
>> Thanks,
>> Jesse
>>
>> On 1/26/2018 12:48 AM, Jacques Le Roux wrote:
>>
>>> Thanks Jinghai,
>>>
>>> That's an interesting information!
>>>
>>> Jacques
>>>
>>>
>>> Le 26/01/2018 à 02:57, Shi Jinghai a écrit :
>>>
 Thanks Mike, I agree with you if such image processing happens
 regularly, perl is a good choice but system may be complicated.

 The image processing speed of JDK 7 is dramatically slower than JDK 6,
 and sometimes wrong, as Kodak and others withdrew their image algorithm
 properties when Java transferred from Sun to Oracle.

 Java will struggle on ImageIO for a while, hope JDK 9 have a big
 improvement. [1]

 Kind Regards,

 Shi Jinghai

 [1] https://bugs.openjdk.java.net/browse/JDK-8041125

 发件人: Mike [mailto:mz4whee...@gmail.com]
 发送时间: 2018年1月26日 0:26
 收件人: user
 抄送: Shi Jinghai
 主题: Re: data import - product images

 A well written perl or shell script can take "original.jpg" in
 directory "x" and create, multiple larger/smaller images with the same
 "root" name. You can use image magic, an image processor package, to do
 this.  Basic stuff for anyone who knows shell scripting.

 You can then import these images using a single XML data_reader file,
 and import it into OFB.  Example:

 >>>productId="798936836182"
 smallImageUrl="/images/products/1/1001MINIGOLF/small.jpg"
 mediumImageUrl="/images/products/1/1001MINIGOLF/medium.jpg"
 largeImageUrl="/images/products/1/1001MINIGOLF/large.jpg"
 originalImageUrl="/images/products/1/1001MINIGOLF/original.jpg"
 detailImageUrl="/images/products/1/1001MINIGOLF/detail.jpg"
 />

 [plus 1000s more].

 A single script can generate the above XML file (with 1000s of
 products), resize 1000s of "original.jpg" into multiple images, and import
 into OFB, all in one pass.  Standard stuff for anyone competent with
 perl/shell scripting.

 What you don't want is make java do busy, CPU intensive work and have
 it render different sizes "on the fly", if that is what you are suggesting.

 On Wed, Jan 24, 2018 at 11:56 PM, Jesse Thomas <
 je...@thomasnetworks.net> wrote:
 Thanks Shi and Rishi for the responses.

 I was hoping to do this the hard way and l learn some new tricks in the
 process. More specifically I was hoping to use a ftl template (like in
 ./specialpurpose/ecommerce/data/DemoTree.xml)

Re: data import - product images

2018-02-23 Thread gil portenseigne


Hello Jesse,

You could manually manage your transactions within your service using :

actualTransaction = TransactionUtil.suspend(); //suspend the current 
transaction

TransactionUtil.begin(600); //Start a new one
[...]
if (ServiceUtil.isError(result)) {
    TransactionUtil.rollback(); //Rollback the new one
} else {
    TransactionUtil.commit(); // Or Commit it
}
TransactionUtil.resume(actualTransaction); //resume the suspended 
transaction.


With this solution you keep your process sync, and you can manage errors.

HTH

Gil


On 23/02/2018 09:29, Jesse Thomas wrote:
I have written a service that checks a folder for image files, resizes 
and updates the Product in OFBiz based on the file name equaling the 
productId. It works for a few hundred images, but when I process a few 
thousand images at the end it hits a transaction timeout. All the 
images are resized and saved. But the product records don't get 
updated. All I'm doing in the database is updating the Product image 
fields, is there a way to force the commit on each product instead at 
the end of the service call?


Thanks,
    Jesse

On 1/26/2018 12:48 AM, Jacques Le Roux wrote:

Thanks Jinghai,

That's an interesting information!

Jacques


Le 26/01/2018 à 02:57, Shi Jinghai a écrit :
Thanks Mike, I agree with you if such image processing happens 
regularly, perl is a good choice but system may be complicated.


The image processing speed of JDK 7 is dramatically slower than JDK 
6, and sometimes wrong, as Kodak and others withdrew their image 
algorithm properties when Java transferred from Sun to Oracle.


Java will struggle on ImageIO for a while, hope JDK 9 have a big 
improvement. [1]


Kind Regards,

Shi Jinghai


[1] https://bugs.openjdk.java.net/browse/JDK-8041125



发件人: Mike [mailto:mz4whee...@gmail.com]
发送时间: 2018年1月26日 0:26
收件人: user
抄送: Shi Jinghai
主题: Re: data import - product images

A well written perl or shell script can take "original.jpg" in 
directory "x" and create, multiple larger/smaller images with the 
same "root" name. You can use image magic, an image processor 
package, to do this.  Basic stuff for anyone who knows shell scripting.


You can then import these images using a single XML data_reader 
file, and import it into OFB.  Example:






[plus 1000s more].



A single script can generate the above XML file (with 1000s of 
products), resize 1000s of "original.jpg" into multiple images, and 
import into OFB, all in one pass.  Standard stuff for anyone 
competent with perl/shell scripting.


What you don't want is make java do busy, CPU intensive work and 
have it render different sizes "on the fly", if that is what you are 
suggesting.


On Wed, Jan 24, 2018 at 11:56 PM, Jesse Thomas 
> wrote:

Thanks Shi and Rishi for the responses.

I was hoping to do this the hard way and l learn some new tricks in 
the process. More specifically I was hoping to use a ftl template 
(like in ./specialpurpose/ecommerce/data/DemoTree.xml) to call a 
service or function directly. Making the solution 100% OFB.


If you know of any examples or clues please let me know.

Thanks again!



On 1/24/2018 11:27 PM, Shi Jinghai wrote:
On generating the other sizes needed, it's already in the source 
code, see
https://github.com/apache/ofbiz-framework/blob/trunk/applications/product/src/main/java/org/apache/ofbiz/product/image/ScaleImage.java 



-邮件原件-
发件人: Jesse Thomas 
[mailto:je...@thomasnetworks.net]

发送时间: 2018年1月24日 15:30
收件人: user@ofbiz.apache.org
主题: data import - product images

Is there is a way of using a data reader to load a product image and 
have it treated as though its uploaded as an "Original Image"
(generating the other sizes needed)? Or whats the best way to load 
images during data migration?


The images are available in the file system and via http, several 
thousand of them. Thanks in advance for any help or advice!


Thanks,

   Jesse

Re: data import - product images

2018-02-23 Thread Swapnil Mane

Hello Jesse,

Solution to avoid transaction timeout is to schedule your job (service)
https://demo-trunk.ofbiz.apache.org/webtools/control/scheduleJob


- Best  Regards,
Swapnil M Mane


On Fri, Feb 23, 2018 at 1:59 PM, Jesse Thomas 
wrote:

> I have written a service that checks a folder for image files, resizes and
> updates the Product in OFBiz based on the file name equaling the productId.
> It works for a few hundred images, but when I process a few thousand images
> at the end it hits a transaction timeout. All the images are resized and
> saved. But the product records don't get updated. All I'm doing in the
> database is updating the Product image fields, is there a way to force the
> commit on each product instead at the end of the service call?
>
> Thanks,
> Jesse
>
>
> On 1/26/2018 12:48 AM, Jacques Le Roux wrote:
>
>> Thanks Jinghai,
>>
>> That's an interesting information!
>>
>> Jacques
>>
>>
>> Le 26/01/2018 à 02:57, Shi Jinghai a écrit :
>>
>>> Thanks Mike, I agree with you if such image processing happens
>>> regularly, perl is a good choice but system may be complicated.
>>>
>>> The image processing speed of JDK 7 is dramatically slower than JDK 6,
>>> and sometimes wrong, as Kodak and others withdrew their image algorithm
>>> properties when Java transferred from Sun to Oracle.
>>>
>>> Java will struggle on ImageIO for a while, hope JDK 9 have a big
>>> improvement. [1]
>>>
>>> Kind Regards,
>>>
>>> Shi Jinghai
>>>
>>>
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8041125
>>>
>>>
>>>
>>> 发件人: Mike [mailto:mz4whee...@gmail.com]
>>> 发送时间: 2018年1月26日 0:26
>>> 收件人: user
>>> 抄送: Shi Jinghai
>>> 主题: Re: data import - product images
>>>
>>> A well written perl or shell script can take "original.jpg" in directory
>>> "x" and create, multiple larger/smaller images with the same "root" name.
>>> You can use image magic, an image processor package, to do this.  Basic
>>> stuff for anyone who knows shell scripting.
>>>
>>> You can then import these images using a single XML data_reader file,
>>> and import it into OFB.  Example:
>>>
>>> 
>>> 
>>> >>productId="798936836182"
>>>smallImageUrl="/images/products/1/1001MINIGOLF/small.jpg"
>>> mediumImageUrl="/images/products/1/1001MINIGOLF/medium.jpg"
>>>largeImageUrl="/images/products/1/1001MINIGOLF/large.jpg"
>>> originalImageUrl="/images/products/1/1001MINIGOLF/original.jpg"
>>> detailImageUrl="/images/products/1/1001MINIGOLF/detail.jpg"
>>> />
>>>
>>> [plus 1000s more].
>>>
>>> 
>>>
>>> A single script can generate the above XML file (with 1000s of
>>> products), resize 1000s of "original.jpg" into multiple images, and import
>>> into OFB, all in one pass.  Standard stuff for anyone competent with
>>> perl/shell scripting.
>>>
>>> What you don't want is make java do busy, CPU intensive work and have it
>>> render different sizes "on the fly", if that is what you are suggesting.
>>>
>>> On Wed, Jan 24, 2018 at 11:56 PM, Jesse Thomas >> > wrote:
>>> Thanks Shi and Rishi for the responses.
>>>
>>> I was hoping to do this the hard way and l learn some new tricks in the
>>> process. More specifically I was hoping to use a ftl template (like in
>>> ./specialpurpose/ecommerce/data/DemoTree.xml) to call a service or
>>> function directly. Making the solution 100% OFB.
>>>
>>> If you know of any examples or clues please let me know.
>>>
>>> Thanks again!
>>>
>>>
>>>
>>> On 1/24/2018 11:27 PM, Shi Jinghai wrote:
>>> On generating the other sizes needed, it's already in the source code,
>>> see
>>> https://github.com/apache/ofbiz-framework/blob/trunk/applica
>>> tions/product/src/main/java/org/apache/ofbiz/product/image/
>>> ScaleImage.java
>>>
>>> -邮件原件-
>>> 发件人: Jesse Thomas [mailto:je...@thomasnetworks.net>> je...@thomasnetworks.net>]
>>> 发送时间: 2018年1月24日 15:30
>>> 收件人: user@ofbiz.apache.org
>>> 主题: data import - product images
>>>
>>> Is there is a way of using a data reader to load a product image and
>>> have it treated as though its uploaded as an "Original Image"
>>> (generating the other sizes needed)? Or whats the best way to load
>>> images during data migration?
>>>
>>> The images are available in the file system and via http, several
>>> thousand of them. Thanks in advance for any help or advice!
>>>
>>> Thanks,
>>>
>>>Jesse
>>>
>>>
>>>
>>
>

Re: data import - product images

2018-02-23 Thread Jesse Thomas

I have written a service that checks a folder for image files, resizes 
and updates the Product in OFBiz based on the file name equaling the 
productId. It works for a few hundred images, but when I process a few 
thousand images at the end it hits a transaction timeout. All the images 
are resized and saved. But the product records don't get updated. All 
I'm doing in the database is updating the Product image fields, is there 
a way to force the commit on each product instead at the end of the 
service call?


Thanks,
    Jesse

On 1/26/2018 12:48 AM, Jacques Le Roux wrote:

Thanks Jinghai,

That's an interesting information!

Jacques


Le 26/01/2018 à 02:57, Shi Jinghai a écrit :
Thanks Mike, I agree with you if such image processing happens 
regularly, perl is a good choice but system may be complicated.


The image processing speed of JDK 7 is dramatically slower than JDK 
6, and sometimes wrong, as Kodak and others withdrew their image 
algorithm properties when Java transferred from Sun to Oracle.


Java will struggle on ImageIO for a while, hope JDK 9 have a big 
improvement. [1]


Kind Regards,

Shi Jinghai


[1] https://bugs.openjdk.java.net/browse/JDK-8041125



发件人: Mike [mailto:mz4whee...@gmail.com]
发送时间: 2018年1月26日 0:26
收件人: user
抄送: Shi Jinghai
主题: Re: data import - product images

A well written perl or shell script can take "original.jpg" in 
directory "x" and create, multiple larger/smaller images with the 
same "root" name. You can use image magic, an image processor 
package, to do this.  Basic stuff for anyone who knows shell scripting.


You can then import these images using a single XML data_reader file, 
and import it into OFB.  Example:






[plus 1000s more].



A single script can generate the above XML file (with 1000s of 
products), resize 1000s of "original.jpg" into multiple images, and 
import into OFB, all in one pass.  Standard stuff for anyone 
competent with perl/shell scripting.


What you don't want is make java do busy, CPU intensive work and have 
it render different sizes "on the fly", if that is what you are 
suggesting.


On Wed, Jan 24, 2018 at 11:56 PM, Jesse Thomas 
> wrote:

Thanks Shi and Rishi for the responses.

I was hoping to do this the hard way and l learn some new tricks in 
the process. More specifically I was hoping to use a ftl template 
(like in ./specialpurpose/ecommerce/data/DemoTree.xml) to call a 
service or function directly. Making the solution 100% OFB.


If you know of any examples or clues please let me know.

Thanks again!



On 1/24/2018 11:27 PM, Shi Jinghai wrote:
On generating the other sizes needed, it's already in the source 
code, see
https://github.com/apache/ofbiz-framework/blob/trunk/applications/product/src/main/java/org/apache/ofbiz/product/image/ScaleImage.java 



-邮件原件-
发件人: Jesse Thomas 
[mailto:je...@thomasnetworks.net]

发送时间: 2018年1月24日 15:30
收件人: user@ofbiz.apache.org
主题: data import - product images

Is there is a way of using a data reader to load a product image and 
have it treated as though its uploaded as an "Original Image"
(generating the other sizes needed)? Or whats the best way to load 
images during data migration?


The images are available in the file system and via http, several 
thousand of them. Thanks in advance for any help or advice!


Thanks,

   Jesse

Re: data import - product images

Re: data import - product images

Re: data import - product images

Re: data import - product images

4 matches

Site Navigation

Mail list logo

Footer information