XML - no encoding
I'm working with a vendor to troubleshoot an error and he's asked me to *not* replace with amp; etc in a URL field. I'm building an XML doc with xmlnew() and then posting it with cfhttp method=post and I'm not explicitly calling urlencodedformat() or anything. CF is just automatically and (I think) correctly performing the encoding. Any way I can stop this from happening just for the duration of the troubleshooting...? -- John Bliss - http://www.linkedin.com/in/jbliss ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:358743 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML - no encoding
CDATA: MyXMLObj.MyXMLChild[1].XMLCData=http://myurl.com?x=ya=b'; Most XML libraries for other languages will read CDATA just like they would a non-escaped node. On Jun 17, 2014, at 2:33 PM, John M Bliss bliss.j...@gmail.com wrote: I'm working with a vendor to troubleshoot an error and he's asked me to *not* replace with amp; etc in a URL field. I'm building an XML doc with xmlnew() and then posting it with cfhttp method=post and I'm not explicitly calling urlencodedformat() or anything. CF is just automatically and (I think) correctly performing the encoding. Any way I can stop this from happening just for the duration of the troubleshooting...? -- John Bliss - http://www.linkedin.com/in/jbliss ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:358744 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML - no encoding
So, using your example, instead of: MyXMLObj.MyXMLChild[1] = 'http://myurl.com?x=ya=b'; ...I'd use: MyXMLObj.MyXMLChild[1].XMLCData = 'http://myurl.com?x=ya=b'; ...? On Tue, Jun 17, 2014 at 2:39 PM, Jon Clausen jon_clau...@silowebworks.com wrote: CDATA: MyXMLObj.MyXMLChild[1].XMLCData=èttp://myurl.com?x=ya=b'; Most XML libraries for other languages will read CDATA just like they would a non-escaped node. On Jun 17, 2014, at 2:33 PM, John M Bliss bliss.j...@gmail.com wrote: I'm working with a vendor to troubleshoot an error and he's asked me to *not* replace with amp; etc in a URL field. I'm building an XML doc with xmlnew() and then posting it with cfhttp method=post and I'm not explicitly calling urlencodedformat() or anything. CF is just automatically and (I think) correctly performing the encoding. Any way I can stop this from happening just for the duration of the troubleshooting...? -- John Bliss - http://www.linkedin.com/in/jbliss ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:358745 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML - no encoding
Exactly. On Jun 17, 2014, at 2:54 PM, John M Bliss bliss.j...@gmail.com wrote: So, using your example, instead of: MyXMLObj.MyXMLChild[1] = 'http://myurl.com?x=ya=b'; ...I'd use: MyXMLObj.MyXMLChild[1].XMLCData = 'http://myurl.com?x=ya=b'; ...? On Tue, Jun 17, 2014 at 2:39 PM, Jon Clausen jon_clau...@silowebworks.com wrote: CDATA: MyXMLObj.MyXMLChild[1].XMLCData=èttp://myurl.com?x=ya=b'; Most XML libraries for other languages will read CDATA just like they would a non-escaped node. On Jun 17, 2014, at 2:33 PM, John M Bliss bliss.j...@gmail.com wrote: I'm working with a vendor to troubleshoot an error and he's asked me to *not* replace with amp; etc in a URL field. I'm building an XML doc with xmlnew() and then posting it with cfhttp method=post and I'm not explicitly calling urlencodedformat() or anything. CF is just automatically and (I think) correctly performing the encoding. Any way I can stop this from happening just for the duration of the troubleshooting...? -- John Bliss - http://www.linkedin.com/in/jbliss ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:358746 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML - no encoding
CDATA: MyXMLObj.MyXMLChild[1].XMLCData=èttp://myurl.com?x=ya=b'; Most XML libraries for other languages will read CDATA just like they would a non-escaped node. I'm pretty sure that this will only work for element values, not attribute values. I'm not sure whether John's using this in an element or an attribute. Dave Watts, CTO, Fig Leaf Software 1-202-527-9569 http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:358747 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML - no encoding
Element. This changed my XML from this: Urlhttp://myurl.com/?x=yamp;a=b/Url ...to this: Url![CDATA[ http://myurl.com/?x=ya=b]]/Url ...which, I guess, was the desired effect. Now, response I'm getting has changed to, URL 'http:// http://myurl.com/?x=ya=b' is not in the expected format. Repeating http://; is new... On Tue, Jun 17, 2014 at 3:03 PM, Dave Watts dwa...@figleaf.com wrote: CDATA: MyXMLObj.MyXMLChild[1].XMLCData=èttp://myurl.com?x=ya=b'; Most XML libraries for other languages will read CDATA just like they would a non-escaped node. I'm pretty sure that this will only work for element values, not attribute values. I'm not sure whether John's using this in an element or an attribute. Dave Watts, CTO, Fig Leaf Software 1-202-527-9569 http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:358748 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML - no encoding
SOLVED: URL had leading space. trim() fixed it. Thanks! :-) On Tue, Jun 17, 2014 at 3:10 PM, John M Bliss bliss.j...@gmail.com wrote: Element. This changed my XML from this: Urlhttp://myurl.com/?x=yamp;a=b http://myurl.com/?x=ya=b/Url ...to this: Url![CDATA[ http://myurl.com/?x=ya=b]]/Url ...which, I guess, was the desired effect. Now, response I'm getting has changed to, URL 'http:// http://myurl.com/?x=ya=b' is not in the expected format. Repeating http://; is new... On Tue, Jun 17, 2014 at 3:03 PM, Dave Watts dwa...@figleaf.com wrote: CDATA: MyXMLObj.MyXMLChild[1].XMLCData=èttp://myurl.com?x=ya=b'; Most XML libraries for other languages will read CDATA just like they would a non-escaped node. I'm pretty sure that this will only work for element values, not attribute values. I'm not sure whether John's using this in an element or an attribute. Dave Watts, CTO, Fig Leaf Software 1-202-527-9569 http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:358749 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Parsing XML response from web services with ColdFusion DeserializeJSON
Hi everyone, I'm working with ColdFusion 9.0 trying to communicate with web services that our state has set up. I use cfhttp tag to submit SOAP request which looks like this: cfhttp method=POST url=http://167.21.60.200/Delaware.ICIS.XmlFiling..Services/Service.svc; useragent=gSOAP/2.8 result=httpResponse cfhttpparam type=header name=content-type value=text/xml cfhttpparam type=header name=charset value=utf-8 cfhttpparam type=header name=content-length value=#len(signOnSoap)# cfhttpparam type=header name=connection value=close cfhttpparam type=header name=accept-encoding value=no-compression / cfhttpparam type=header name=SOAPAction value=Delaware.ICIS.XmlFiling.Services/FilingService/FileCheckAgentAccountBalanceImmediate cfhttpparam type=xml value=#trim(ckBalRequest)# /cfhttp Where ckBalRequest is an xml variable created according to the web service provider specifications. I have no problem getting the request to them and receiving the response. However, while the response should be an XML variable as well, when I tried to use XMLParse(httpReponse.FileContent), CF gave me an error: An error occured while Parsing an XML document. Content is not allowed in prolog. So, tried cfdump to see the response I was getting. Then I got the following error: java.io.ByteArrayOutputStream. While researching about this error, I came upon suggestion to use DeserializeJSON function for the response. Now I received this error which is something within the response: JSON parsing failure at character 2:'-' in --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 Content-ID: http://tempuri.org/0 Content-Transfer-Encoding: 8bit Content-Type: application/xop+xml;charset=utf-8;type=text/xml s:Envelope xmlns:s=http://schemas.xmlsoap.org/soap/envelope/;s:Headerh:agentNumber xmlns:h=Delaware.ICIS.XmlFiling.Services9020288/h:agentNumberh:agentPONumber xmlns:h=Delaware.ICIS.XmlFiling.ServicesNoPOforThisTrans/h:agentPONumberh:attentionLine xmlns:h=Delaware.ICIS.XmlFiling.ServicesAttn: Dan's Listner/h:attentionLineh:fileDateTime xmlns:h=Delaware.ICIS.XmlFiling.Services20130509T11:25:50:9196/h:fileDateTimeh:packetNumber xmlns:h=Delaware.ICIS.XmlFiling.Services9020288_20130509T1125420351/h:packetNumberh:receivedDateTime xmlns:h=Delaware.ICIS.XmlFiling.Services20130509T11:25:50:9196/h:receivedDateTimeh:successful xmlns:h=Delaware.ICIS.XmlFiling.Servicestrue/h:successful/s:Headers:BodycheckAgentAccountBalanceResponse xmlns=Delaware.ICIS..XmlFiling.ServicesaccountBalanc... Here the actual response that the state's web service is sending me, but ColdFusion keeps having trouble seeing it as a structure: --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 Content-ID: http://tempuri.org/0 Content-Transfer-Encoding: 8bit Content-Type: application/xop+xml;charset=utf-8;type=text/xml s:Envelope xmlns:s=http://schemas.xmlsoap.org/soap/envelope/; s:Header h:agentNumber xmlns:h=Delaware.ICIS.XmlFiling.Services9020288/h:agentNumber h:agentPONumber xmlns:h=Delaware.ICIS.XmlFiling.ServicesNoPOforThisTrans/h:agentPONumber h:attentionLine xmlns:h=Delaware.ICIS.XmlFiling.ServicesAttn: Dan's Listner/h:attentionLine h:fileDateTime xmlns:h=Delaware.ICIS.XmlFiling.Services20130509T11:25:50:9196/h:fileDateTime h:packetNumber xmlns:h=Delaware.ICIS.XmlFiling.Services9020288_20130107T1623410102/h:packetNumber h:receivedDateTime xmlns:h=Delaware.ICIS.XmlFiling.Services20130107T16:23:41:3116/h:receivedDateTime h:successful xmlns:h=Delaware.ICIS.XmlFiling.Servicestrue/h:successful /s:Header s:Body checkAgentAccountBalanceResponse xmlns=Delaware.ICIS.XmlFiling.Services accountBalance797250.4162/accountBalance fundsReserved254873.2338/fundsReserved availableBalance542377.1824/availableBalance /checkAgentAccountBalanceResponse /s:Body /s:Envelope --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 When I simply display the response by placing on the page cfoutput#httpResponse.FileContent#/cfoutput, I get this: --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 Content-ID: Content-Transfer-Encoding: 8bit Content-Type: application/xop+xml;charset=utf-8;type=text/xml 9020288NoPOforThisTransAttn: Dan's Listner20130509T11:25:50:91969020288_20130509T112542035120130509T11:25:50:9196true789222.1662261386.2338527835.9324 --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34-- As you can see all of the headers are compressed then, and it's nearly impossible to extract the values. So I need to have the response in a structure that would give me access to the values (and so that I know what they are for me to use them correctly). So far DeserializeJSON was the only function that actually displayed headers in somewhat XML structure, but it has trouble with unique identifier (which is a unique identifier for the session) in the beginning of response. Simply trying to remove it from the response by using ColdFusion's Replace() isn't working at all. It's still there. I'm
RE: Parsing XML response from web services with ColdFusion DeserializeJSON
Hello Sasha. Try this... cfset results = reReplace(httpReponse.FileContent, ^[^]*, , ALL) cfset results = xmlParse(results) cfdump var=#results# Hope this helps... Che ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:355685 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Parsing XML response from web services with ColdFusion DeserializeJSON
I'm working with ColdFusion 9.0 trying to communicate with web services that our state has set up. I use cfhttp tag to submit SOAP request which looks like this: cfhttp method=POST url=http://167.21.60.200/Delaware.ICIS.XmlFiling..Services/Service.svc; useragent=gSOAP/2.8 result=httpResponse cfhttpparam type=header name=content-type value=text/xml cfhttpparam type=header name=charset value=utf-8 cfhttpparam type=header name=content-length value=#len(signOnSoap)# cfhttpparam type=header name=connection value=close cfhttpparam type=header name=accept-encoding value=no-compression / cfhttpparam type=header name=SOAPAction value=Delaware.ICIS.XmlFiling.Services/FilingService/FileCheckAgentAccountBalanceImmediate cfhttpparam type=xml value=#trim(ckBalRequest)# /cfhttp My first question is, have you tried using CFINVOKE to call the web service? That doesn't always work, but it's specifically intended to invoke SOAP services. Where ckBalRequest is an xml variable created according to the web service provider specifications. I have no problem getting the request to them and receiving the response. However, while the response should be an XML variable as well, when I tried to use XMLParse(httpReponse.FileContent), CF gave me an error: An error occured while Parsing an XML document. Content is not allowed in prolog. So, tried cfdump to see the response I was getting. Then I got the following error: java.io.ByteArrayOutputStream. While researching about this error, I came upon suggestion to use DeserializeJSON function for the response. Now I received this error which is something within the response: JSON parsing failure at character 2:'-' in --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 ... Here the actual response that the state's web service is sending me, but ColdFusion keeps having trouble seeing it as a structure: --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 Content-ID: http://tempuri.org/0 Content-Transfer-Encoding: 8bit Content-Type: application/xop+xml;charset=utf-8;type=text/xml s:Envelope xmlns:s=http://schemas.xmlsoap.org/soap/envelope/; s:Header h:agentNumber xmlns:h=Delaware.ICIS.XmlFiling.Services9020288/h:agentNumber h:agentPONumber xmlns:h=Delaware.ICIS.XmlFiling.ServicesNoPOforThisTrans/h:agentPONumber h:attentionLine xmlns:h=Delaware.ICIS.XmlFiling.ServicesAttn: Dan's Listner/h:attentionLine h:fileDateTime xmlns:h=Delaware.ICIS.XmlFiling.Services20130509T11:25:50:9196/h:fileDateTime h:packetNumber xmlns:h=Delaware.ICIS.XmlFiling.Services9020288_20130107T1623410102/h:packetNumber h:receivedDateTime xmlns:h=Delaware.ICIS.XmlFiling.Services20130107T16:23:41:3116/h:receivedDateTime h:successful xmlns:h=Delaware.ICIS.XmlFiling.Servicestrue/h:successful /s:Header s:Body checkAgentAccountBalanceResponse xmlns=Delaware.ICIS.XmlFiling.Services accountBalance797250.4162/accountBalance fundsReserved254873.2338/fundsReserved availableBalance542377.1824/availableBalance /checkAgentAccountBalanceResponse /s:Body /s:Envelope --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 When I simply display the response by placing on the page cfoutput#httpResponse.FileContent#/cfoutput, I get this: --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 Content-ID: Content-Transfer-Encoding: 8bit Content-Type: application/xop+xml;charset=utf-8;type=text/xml 9020288NoPOforThisTransAttn: Dan's Listner20130509T11:25:50:91969020288_20130509T112542035120130509T11:25:50:9196true789222. 1662261386.2338527835.9324 --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34-- As you can see all of the headers are compressed then, and it's nearly impossible to extract the values. So I need to have the response in a structure that would give me access to the values (and so that I know what they are for me to use them correctly). The reason all of the headers are compressed then is that the browser isn't going to show you tags that it doesn't know how to parse, and you're not outputting a well-formed XML document. There are several problems here. First, it appears that your response is part of a MIME multipart message, and you're including the message headers and the boundary markers (the lines that denote the beginning and end of the part - the two lines at the very top and bottom that contain --uuid ...). You need to remove those lines, and the first three lines after the first boundary marker which are part of the response header and not the response body. Those lines are the content that's not allowed in the prolog - the prolog is the part before your root XML element, and typically contains a DTD if it exists at all. After that - when you just have the parts that begin and end with s:Envelope - you'll have a well-formed XML document. You can probably parse that just fine with XMLParse. So far DeserializeJSON was the only function that actually displayed headers
RE: Parsing XML response from web services with ColdFusion DeserializeJSON
Hi Che, Thank you for your advice, but unfortunately it didn't help. Here is an error I got: An error occured while Parsing an XML document. Element or attribute do not match QName production: QName::=(NCName':')?NCName. Sincerely, Sasha. -Original Message- From: Che Vilnonis [mailto:ch...@asitv.com] Sent: Thursday, May 09, 2013 1:04 PM To: cf-talk Subject: RE: Parsing XML response from web services with ColdFusion DeserializeJSON Hello Sasha. Try this... cfset results = reReplace(httpReponse.FileContent, ^[^]*, , ALL) cfset results = xmlParse(results) cfdump var=#results# Hope this helps... Che ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:355688 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Parsing XML response from web services with ColdFusion DeserializeJSON
I'm working with ColdFusion 9.0 trying to communicate with web services that our state has set up. I use cfhttp tag to submit SOAP request which looks like this: cfhttp method=POST url=http://167.21.60.200/Delaware.ICIS. XmlFiling..Services/Service.svc useragent=gSOAP/2.8 result=httpResponse cfhttpparam type=header name=content-type value=text/xml cfhttpparam type=header name=charset value=utf-8 cfhttpparam type=header name=content-length value=#len(signOnSoap)# cfhttpparam type=header name=connection value=close cfhttpparam type=header name=accept-encoding value=no-compression / cfhttpparam type=header name=SOAPAction value=Delaware. ICIS.XmlFiling. Services/FilingService/FileCheckAgentAccountBalanceImmediate cfhttpparam type=xml value=#trim(ckBalRequest)# /cfhttp My first question is, have you tried using CFINVOKE to call the web service? That doesn't always work, but it's specifically intended to invoke SOAP services. Where ckBalRequest is an xml variable created according to the web service provider specifications. I have no problem getting the request to them and receiving the response. However, while the response should be an XML variable as well, when I tried to use XMLParse(httpReponse.FileContent), CF gave me an error: An error occured while Parsing an XML document. Content is not allowed in prolog. So, tried cfdump to see the response I was getting. Then I got the following error: java.io.ByteArrayOutputStream. While researching about this error, I came upon suggestion to use DeserializeJSON function for the response. Now I received this error which is something within the response: JSON parsing failure at character 2:'-' in --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 ... Here the actual response that the state's web service is sending me, but ColdFusion keeps having trouble seeing it as a structure: --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 Content-ID: http://tempuri.org/0 Content-Transfer-Encoding: 8bit Content-Type: application/xop+xml;charset=utf-8;type=text/xml s:Envelope xmlns:s=http://schemas.xmlsoap.org/soap/envelope/; s:Header h:agentNumber xmlns:h=Delaware.ICIS.XmlFiling. Services9020288/h:agentNumber h:agentPONumber xmlns:h=Delaware.ICIS.XmlFiling. ServicesNoPOforThisTrans/h:agentPONumber h:attentionLine xmlns:h=Delaware.ICIS.XmlFiling.ServicesAttn: Dan's Listner/h:attentionLine h:fileDateTime xmlns:h=Delaware.ICIS.XmlFiling. Services20130509T11:25:50:9196/h:fileDateTime h:packetNumber xmlns:h=Delaware.ICIS.XmlFiling. Services9020288_20130107T1623410102/h:packetNumber h:receivedDateTime xmlns:h=Delaware.ICIS.XmlFiling. Services20130107T16:23:41:3116/h:receivedDateTime h:successful xmlns:h=Delaware.ICIS.XmlFiling. Servicestrue/h:successful /s:Header s:Body checkAgentAccountBalanceResponse xmlns=Delaware.ICIS.XmlFiling. Services accountBalance797250.4162/accountBalance fundsReserved254873.2338/fundsReserved availableBalance542377.1824/availableBalance /checkAgentAccountBalanceResponse /s:Body /s:Envelope --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 When I simply display the response by placing on the page cfoutput#httpResponse.FileContent#/cfoutput, I get this: --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34 Content-ID: Content-Transfer-Encoding: 8bit Content-Type: application/xop+xml;charset=utf-8;type=text/xml 9020288NoPOforThisTransAttn: Dan's Listner20130509T11 :25:50:91969020288_20130509T112542035120130509T11:25:50:9196true789222. 1662261386.2338527835.9324 --uuid:e2f0c19a-df5f-4570-88f2-b641a09178ee+id=34-- As you can see all of the headers are compressed then, and it's nearly impossible to extract the values. So I need to have the response in a structure that would give me access to the values (and so that I know what they are for me to use them correctly). The reason all of the headers are compressed then is that the browser isn't going to show you tags that it doesn't know how to parse, and you're not outputting a well-formed XML document. There are several problems here. First, it appears that your response is part of a MIME multipart message, and you're including the message headers and the boundary markers (the lines that denote the beginning and end of the part - the two lines at the very top and bottom that contain --uuid ...). You need to remove those lines, and the first three lines after the first boundary marker which are part of the response header and not the response body. Those lines are the content that's not allowed in the prolog - the prolog is the part before your root XML element, and typically contains a DTD if it exists at all. After that - when you just have the parts that begin and end with s:Envelope - you'll have a well-formed XML document. You can probably parse that just fine
Re: Parsing XML response from web services with ColdFusion DeserializeJSON
Next, I do realize that I need to remove the two lines at the very top and bottom that contain --uuid ...) and the first three lines after the first boundary marker which are part of the response header. The problem here is that simply using Replace() function to replace those lines with nothing isn't working. The Replace function requires that you specify a pattern. There are a couple of other ways you could do this: - replace everything before and after s:Envelope using simple string functions - treat the response as a list of lines and use list functions to remove the lines you don't want. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:355694 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Parsing XML response from web services with ColdFusion DeserializeJSON
I had very similar problems with the BING AdCenter API which I and several others never could get working. Almost identical errors. I even sent over the raw responses from Fiddler and Microsoft technical support response to me was the same thing. It's one of CF's downfalls. Now I'm going to have to go back try again :) Round two. Matt -Original Message- From: Dave Watts [mailto:dwa...@figleaf.com] Sent: Thursday, May 09, 2013 12:58 PM To: cf-talk Subject: Re: Parsing XML response from web services with ColdFusion DeserializeJSON Next, I do realize that I need to remove the two lines at the very top and bottom that contain --uuid ...) and the first three lines after the first boundary marker which are part of the response header. The problem here is that simply using Replace() function to replace those lines with nothing isn't working. The Replace function requires that you specify a pattern. There are a couple of other ways you could do this: - replace everything before and after s:Envelope using simple string functions - treat the response as a list of lines and use list functions to remove the lines you don't want. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:355695 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
neo-runtime xml
Hi Guys, (Sorry I may have already posted this on my really old login so aplogies for the repost) We had to reboot an old Win2K box which is running MX 6.1. Since the reboot all CF pages are rending as white pages, no source/markup etc. Including /cfide/Administrator. When looking at the server.log we see the following: Information,main,12/31/12,11:12:43,,Starting license... Information,main,12/31/12,11:12:43,,Starting License server ... Information,main,12/31/12,11:12:43,,Starting scheduler... Information,main,12/31/12,11:12:43,,Starting debugging... Information,main,12/31/12,11:12:43,,Starting sql... Information,main,12/31/12,11:12:43,,Pool Manager Started Information,main,12/31/12,11:12:44,,Starting mail... Error,main,12/31/12,11:12:44,,[C:\CFusionMX\lib\neo-runtime.xml]java.io.FileNotFoundException: C:\CFusionMX\lib\neo-runtime.xml (The system cannot find the file specified) Information,main,12/31/12,11:12:44,,Starting cron... Information,main,12/31/12,11:12:45,,Starting registry... Information,main,12/31/12,11:12:45,,Starting client... Error,main,12/31/12,11:12:45,,The Runtime service is not available. This exception is usually caused by service startup failure. Please check your server configuration. Information,main,12/31/12,11:12:45,,Starting xmlrpc... Information,main,12/31/12,11:12:45,,Starting graphing... So it looks like we may have lost the neo-runtime.xml file during that reboot and don't have a backup. Does anybody have as near a default neo-runtime.xml file or know how we could get one without a CF reinstall. Or know of a way to repair this quickly. Regards, Ian. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:353724 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: neo-runtime xml
all those neo files get backed up automatically when they change, so look for a backup copy in the same folder. I would also strongly suggest you consider taking backups of your server. On Mon, Dec 31, 2012 at 12:37 PM, Ian Chapman ian.chap...@melodimedia.co.uk wrote: Hi Guys, (Sorry I may have already posted this on my really old login so aplogies for the repost) We had to reboot an old Win2K box which is running MX 6.1. Since the reboot all CF pages are rending as white pages, no source/markup etc. Including /cfide/Administrator. When looking at the server.log we see the following: Information,main,12/31/12,11:12:43,,Starting license... Information,main,12/31/12,11:12:43,,Starting License server ... Information,main,12/31/12,11:12:43,,Starting scheduler... Information,main,12/31/12,11:12:43,,Starting debugging... Information,main,12/31/12,11:12:43,,Starting sql... Information,main,12/31/12,11:12:43,,Pool Manager Started Information,main,12/31/12,11:12:44,,Starting mail... Error,main,12/31/12,11:12:44,,[C:\CFusionMX\lib\neo-runtime.xml]java.io.FileNotFoundException: C:\CFusionMX\lib\neo-runtime.xml (The system cannot find the file specified) Information,main,12/31/12,11:12:44,,Starting cron... Information,main,12/31/12,11:12:45,,Starting registry... Information,main,12/31/12,11:12:45,,Starting client... Error,main,12/31/12,11:12:45,,The Runtime service is not available. This exception is usually caused by service startup failure. Please check your server configuration. Information,main,12/31/12,11:12:45,,Starting xmlrpc... Information,main,12/31/12,11:12:45,,Starting graphing... So it looks like we may have lost the neo-runtime.xml file during that reboot and don't have a backup. Does anybody have as near a default neo-runtime.xml file or know how we could get one without a CF reinstall. Or know of a way to repair this quickly. Regards, Ian. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:353726 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
duplicate keys in rss xml feed
Hi All - I am parsing rss feed. The feed has custom data and custom data can have duplicate attributes. I am following the example at http://www.raymondcamden.com/index.cfm/2008/4/23/Ask-a-Jedi-Handling-RSS-feeds-with-custom-data. For example if the feed has following data, and the last custom attribute npr:rmaudio is defined twice but with different values. How can I get those two values. Currently, I am able to get the first value only. item titlePennsylvania Primary Roundup/title descriptionBarack Obama improved his showing among white, middle-class voters, but not enough to beat Hillary Clinton in the Pennsylvania primary on Tuesday. NPR's National Political Correspondent Mara Liasson analyzes the race with Robert Seigel./description pubDateTue, 22 Apr 2008 21:53:01 -0400/pubDate linkhttp://www.npr.org/stations/force/force_localization.php?station=KSJD_FMamp;url=http://www.npr.org/templates/story/story.php?storyId=89862645amp;ft=1amp;f=2/link guidhttp://www.npr.org/templates/story/story.php?storyId=89862645amp;ft=1amp;f=2/guid npr:wmaudiohttp://www.npr.org/templates/dmg/dmg_wmref_em.php?id=89863241amp;type=1amp;date=22-Apr-2008amp;mtype=WM/npr:wmaudio npr:rmaudiohttp://www.npr.org/templates/dmg/dmg_rpm.rpm?id=89863241amp;type=1amp;date=22-Apr-2008amp;mtype=RM/npr:rmaudio npr:rmaudiohttp://www.npr.org/templates/dmg/dmg_rpm.rpm?id=89863241amp;type=21amp;date=22-Apr-2008amp;mtype=RM/npr:rmaudio /item ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:353366 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: duplicate keys in rss xml feed
It should be available as an array under the node. Can you share the full URL of the feed? If so I can make a demo for you. On Wed, Dec 5, 2012 at 2:41 PM, fun and learning funandlrnn...@gmail.comwrote: item titlePennsylvania Primary Roundup/title descriptionBarack Obama improved his showing among white, middle-class voters, but not enough to beat Hillary Clinton in the Pennsylvania primary on Tuesday. NPR's National Political Correspondent Mara Liasson analyzes the race with Robert Seigel./description pubDateTue, 22 Apr 2008 21:53:01 -0400/pubDate link http://www.npr.org/stations/force/force_localization.php?station=KSJD_FMamp;url=http://www.npr.org/templates/story/story.php?storyId=89862645amp;ft=1amp;f=2 /link guid http://www.npr.org/templates/story/story.php?storyId=89862645amp;ft=1amp;f=2 /guid npr:wmaudio http://www.npr.org/templates/dmg/dmg_wmref_em.php?id=89863241amp;type=1amp;date=22-Apr-2008amp;mtype=WM /npr:wmaudio npr:rmaudio http://www.npr.org/templates/dmg/dmg_rpm.rpm?id=89863241amp;type=1amp;date=22-Apr-2008amp;mtype=RM /npr:rmaudio npr:rmaudio http://www.npr.org/templates/dmg/dmg_rpm.rpm?id=89863241amp;type=21amp;date=22-Apr-2008amp;mtype=RM /npr:rmaudio /item -- === Raymond Camden, Adobe Developer Evangelist Email : raymondcam...@gmail.com Blog : www.raymondcamden.com Twitter: cfjedimaster ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:353368 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: duplicate keys in rss xml feed
Ray, Thanks for the suggestion on keys being available as array under the node. I was able to get the values for duplicate keys. Below is the code I used, I guess it can be improved upon cfloop index=x from=1 to=#arrayLen(xmlResult.rss.channel.item)# cfset valueList = cfset count = 0 cfset idx = xmlResult.rss.channel.item[x] cfset keys = #StructKeyList(idx)# cfset keys = ListSort(keys,textnocase,asc,,) cfloop list=#keys# index=key cfif key EQ npr:rmaudio cfset count = count + 1 cfif valueList EQ cfset valueList = #idx[key][count].XmlText# cfelse cfset valueList = valueList , #idx[key][count].XmlText# /cfif /cfif /cfloop /cfloop On Wed, Dec 5, 2012 at 2:54 PM, Raymond Camden raymondcam...@gmail.comwrote: It should be available as an array under the node. Can you share the full URL of the feed? If so I can make a demo for you. On Wed, Dec 5, 2012 at 2:41 PM, fun and learning funandlrnn...@gmail.com wrote: item titlePennsylvania Primary Roundup/title descriptionBarack Obama improved his showing among white, middle-class voters, but not enough to beat Hillary Clinton in the Pennsylvania primary on Tuesday. NPR's National Political Correspondent Mara Liasson analyzes the race with Robert Seigel./description pubDateTue, 22 Apr 2008 21:53:01 -0400/pubDate link http://www.npr.org/stations/force/force_localization.php?station=KSJD_FMamp;url=http://www.npr.org/templates/story/story.php?storyId=89862645amp;ft=1amp;f=2 /link guid http://www.npr.org/templates/story/story.php?storyId=89862645amp;ft=1amp;f=2 /guid npr:wmaudio http://www.npr.org/templates/dmg/dmg_wmref_em.php?id=89863241amp;type=1amp;date=22-Apr-2008amp;mtype=WM /npr:wmaudio npr:rmaudio http://www.npr.org/templates/dmg/dmg_rpm.rpm?id=89863241amp;type=1amp;date=22-Apr-2008amp;mtype=RM /npr:rmaudio npr:rmaudio http://www.npr.org/templates/dmg/dmg_rpm.rpm?id=89863241amp;type=21amp;date=22-Apr-2008amp;mtype=RM /npr:rmaudio /item -- === Raymond Camden, Adobe Developer Evangelist Email : raymondcam...@gmail.com Blog : www.raymondcamden.com Twitter: cfjedimaster ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:353369 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML Security
I was presented with some questions regarding XML and was wondering if there are any setting in Coldfusion to disable any of these or I do not need to worry about it since we do not use any XML in our code: 1. How application employs methods for XML schema validation. 2. How application disables use of inline XML Document Type Definition (DTD) schemas in XML parsing objects. 3. How application manages DTD parsing behavior as a key to preventing the invocation of XML bombs. Is it safe to say IF we were to use XML we could use the XML validation function built in CF9? Honestly, I don't know whether CF's validation would prevent any of these issues. CF's parser does allow you to declare DTD entities directly within an XML document, I think. You'd need to pre-parse the document yourself to prevent this, perhaps. But since your application doesn't use XML, your best bet is to say just that, and no more. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsi ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:350660 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
XML Security
I was presented with some questions regarding XML and was wondering if there are any setting in Coldfusion to disable any of these or I do not need to worry about it since we do not use any XML in our code: 1. How application employs methods for XML schema validation. 2. How application disables use of inline XML Document Type Definition (DTD) schemas in XML parsing objects. 3. How application manages DTD parsing behavior as a key to preventing the invocation of XML bombs. Is it safe to say IF we were to use XML we could use the XML validation function built in CF9? ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:350644 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML Security
That's correct Chad. If you run linux or strict casing this is also a security improvement over loose DTD as well. It eerily close to SOX compliance questions that auditors generally ask when reviewing apps. On Thu, Apr 5, 2012 at 8:52 AM, Chad Baloga cbal...@gmail.com wrote: I was presented with some questions regarding XML and was wondering if there are any setting in Coldfusion to disable any of these or I do not need to worry about it since we do not use any XML in our code: 1. How application employs methods for XML schema validation. 2. How application disables use of inline XML Document Type Definition (DTD) schemas in XML parsing objects. 3. How application manages DTD parsing behavior as a key to preventing the invocation of XML bombs. Is it safe to say IF we were to use XML we could use the XML validation function built in CF9? ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:350645 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML and unicode: 0x1a
I don't want to sound stupid but how does regex work? I could use something similar to this and always fall flat on my face every time someone mentions regex Ken Hammond IT Director The Salem Group Phone: 630-873-3018 Fax: 630-932-7010 Email: khamm...@saleminc.com www.saleminc.com On Feb 23, 2012, at 1:04 PM, Leigh wrote: Have you tried a regex replace of \x1a ? It *might* work. Worth a shot anyway. -Leigh ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:350187 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
XML and unicode: 0x1a
Apparently, after a billion hours of trying to figure out why I couldn't successfully output some xml data, it seems that the unicode: 0x1a character is contained in a text string and making it fail. #xmlformat(thestring)# does not successfully strip it from the string. How can I get rid of that character? Must be some sort of replace(thestring,'0x1a','','all') thing but i can't figure out how to reference it. Thanks in advance for any help, Les ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:350072 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML and unicode: 0x1a
Have you tried a regex replace of \x1a ? It *might* work. Worth a shot anyway. -Leigh ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:350079 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML and unicode: 0x1a
On 2/24/2012 2:04 AM, Leigh wrote: Have you tried a regex replace of \x1a ? It *might* work. Worth a shot anyway. and there are several other codepoints that are illegal in XML. probably should sweep them all away. these are supposed to be a-ok for XML: #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10] /* any Unicode character, excluding the surrogate blocks, FFFE, and . */ http://www.w3.org/TR/REC-xml/#charsets so maybe this (probably wrong, regex isn't my strong suit): [^\x09\x0A\x0D\x20-\xD7FF\xE000-\xFFFD\x1-x10] - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1913 / Virus Database: 2114/4827 - Release Date: 02/23/12 ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:350085 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Is what I'm trying to do with XML even possible?
Hi, all... I've been banging my head against this XML wall for over a week now and either I just don't get it or what I'm trying to do is not possible given what the vendor is providing. The vendor of product info provides 3 ways to access product information: Categories, Product within those Categories, and each Product's details Now, if I were pulling this info from a db, I would naturally use loops to get the categories, products within each categories and products details for each product. However I can't see to get this approach to work with info coming from cfhttp links. For obtaining all categories, the vendor provides: http://lennox.com/api/v1/z7RRSHM/Categories/ For obtaining all products within categories: http://lennox.com/api/v1/z7RRSHM/Category/CategoryID And for product details: http://lennox.com/api/v1/z7RRSHM/Product/productID I've tried to set up all kinds of loops to get this info at this same time while looping the cfhttp queries. Is this even possible given the way the info is provided? You can see what's being return at this links above. To get this kind of display: Category 1 - product one: This is a great product! - product two: This another another great product! - product three: Yet another great product! Category 2 - product one: A super category 2 product! - product two: Another great category 2 product! - product three: Wow! Another super category 2 product! etc, etc. In order to get that kind of looping over each category and product list, do I need to turn to putting the info into a database first? Or is there a way using the cfhttp statements that vendor is providing? I want this info integrated in how I get it so I can use a jQuery slide hide form of product/product details display for a particular category, rather than a traditional, - show all categories - click on a category - show all products - click on a product for details - start over again Thanks for ANY clues! I know this will all be so clear and easy one day, but for now, I'm just lost in the sauce! Perhaps someone has a wish list somewhere on Amazon or someplace that needs some action?! :o) Rick ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349306 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Is what I'm trying to do with XML even possible?
Rick, does something like this work for you? cfset theURL = http://www.lennox.com/api/v1/z7RRSHM/Categories; cfhttp url=#theURL# method=GET result=xmlFeed cfset xmlFeed = xmlParse(theURL) cfset theArray = xmlSearch(xmlFeed, //*[local-name() = 'Category']) / cfloop index=i from=1 to=#arrayLen(theArray)# cfoutput Category Name = #replace(theArray[i].CategoryName.xmlText, : , , ALL)#br /cfoutput /cfloop Hope that helps, Che -Original Message- From: Rick Faircloth [mailto:r...@whitestonemedia.com] Sent: Wednesday, January 04, 2012 10:38 AM To: cf-talk Subject: Is what I'm trying to do with XML even possible? Hi, all... I've been banging my head against this XML wall for over a week now and either I just don't get it or what I'm trying to do is not possible given what the vendor is providing. The vendor of product info provides 3 ways to access product information: Categories, Product within those Categories, and each Product's details Now, if I were pulling this info from a db, I would naturally use loops to get the categories, products within each categories and products details for each product. However I can't see to get this approach to work with info coming from cfhttp links. For obtaining all categories, the vendor provides: http://lennox.com/api/v1/z7RRSHM/Categories/ ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349307 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Is what I'm trying to do with XML even possible?
Is this even possible given the way the info is provided? In order to get that kind of looping over each category and product list, do I need to turn to putting the info into a database first? Or is there a way using the cfhttp statements that vendor is providing? Yes, it's definitely possible. However, you probably don't want to do it all at runtime, if information doesn't change that frequently. For example, I doubt that the categories change every day. So, here's what I'd do. 1. Fetch the XML files periodically, based on their Last-Modified date if they have one. 2. Read the Categories XML file as an XML document, then use /Categories/Category as an XPath query to find your Category nodeset - you should then be able to loop over that. 3. For each category ID attribute, read the appropriate Product XML file as an XML document, then use the XPath query /Products/SubCategory/Product (or maybe just //Product, since I guess it's possible that not all products have subcategories) - this again will return a nodeset that you can loop over. 4. Display stuff. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349308 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Is what I'm trying to do with XML even possible?
In trying to achieve step number 1, I tried to create a document on my hard drive. Should this code not create an xml document on my e: drive? cfscript URLToPull = 'http://lennox.com/api/v1/z7RRSHM/categories/'; /cfscript cfhttp url = '#URLToPull#' method = 'get' timeout = '60' /cfhttp cfset myXMLDocument = XmlParse(cfhttp.fileContent) cfset myXMLDocument = toString(myXMLDocument) cffile action=write file=e:\temp\myxmldoc.xml output=#myXMLDocument# -Original Message- From: Dave Watts [mailto:dwa...@figleaf.com] Sent: Wednesday, January 04, 2012 10:52 AM To: cf-talk Subject: Re: Is what I'm trying to do with XML even possible? Is this even possible given the way the info is provided? In order to get that kind of looping over each category and product list, do I need to turn to putting the info into a database first? Or is there a way using the cfhttp statements that vendor is providing? Yes, it's definitely possible. However, you probably don't want to do it all at runtime, if information doesn't change that frequently. For example, I doubt that the categories change every day. So, here's what I'd do. 1. Fetch the XML files periodically, based on their Last-Modified date if they have one. 2. Read the Categories XML file as an XML document, then use /Categories/Category as an XPath query to find your Category nodeset - you should then be able to loop over that. 3. For each category ID attribute, read the appropriate Product XML file as an XML document, then use the XPath query /Products/SubCategory/Product (or maybe just //Product, since I guess it's possible that not all products have subcategories) - this again will return a nodeset that you can loop over. 4. Display stuff. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349309 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Is what I'm trying to do with XML even possible?
Why do you xmlparse and then convert it back to a string? It was a string originally. Just save cfhttp.filecontent. Actually, cfhttp supports saving it immediately. On Wed, Jan 4, 2012 at 11:15 AM, Rick Faircloth r...@whitestonemedia.com wrote: In trying to achieve step number 1, I tried to create a document on my hard drive. Should this code not create an xml document on my e: drive? cfscript URLToPull = 'http://lennox.com/api/v1/z7RRSHM/categories/'; /cfscript cfhttp url = '#URLToPull#' method = 'get' timeout = '60' /cfhttp cfset myXMLDocument = XmlParse(cfhttp.fileContent) cfset myXMLDocument = toString(myXMLDocument) cffile action=write file=e:\temp\myxmldoc.xml output=#myXMLDocument# -Original Message- From: Dave Watts [mailto:dwa...@figleaf.com] Sent: Wednesday, January 04, 2012 10:52 AM To: cf-talk Subject: Re: Is what I'm trying to do with XML even possible? Is this even possible given the way the info is provided? In order to get that kind of looping over each category and product list, do I need to turn to putting the info into a database first? Or is there a way using the cfhttp statements that vendor is providing? Yes, it's definitely possible. However, you probably don't want to do it all at runtime, if information doesn't change that frequently. For example, I doubt that the categories change every day. So, here's what I'd do. 1. Fetch the XML files periodically, based on their Last-Modified date if they have one. 2. Read the Categories XML file as an XML document, then use /Categories/Category as an XPath query to find your Category nodeset - you should then be able to loop over that. 3. For each category ID attribute, read the appropriate Product XML file as an XML document, then use the XPath query /Products/SubCategory/Product (or maybe just //Product, since I guess it's possible that not all products have subcategories) - this again will return a nodeset that you can loop over. 4. Display stuff. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349310 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Is what I'm trying to do with XML even possible?
In trying to achieve step number 1, I tried to create a document on my hard drive. Should this code not create an xml document on my e: drive? All you need to fetch a file is a single CFHTTP with the PATH and FILE attributes. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349311 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: XML! Aaargh! How do I do this?
Hopefully, the error of my ways will be obvious to thos
of you who work with xml a lot, but...HELP!!!
Given this demo xml:
?xml version=1.0?
Products
Category ID=1Air Conditioners/Category
SubCategory ID=1
SubCategoryNameAir Conditioners/SubCategoryName
ShortDescription/
LongDescription/
Product ID=XC21 New=N EnergyStar=Y SortOrder=100
ModelNumberXC21/ModelNumber
ModelNameXC21 Air Conditioner/ModelName
OneLinerThe most quiet and efficient central air
conditioner you can buy/OneLiner
Ratings
Rating Type=SEERup to 20.5/Rating
Rating Type=Sound69/Rating
/Ratings
SmallImagehttp://www.lennox.com/res/photos/44383_small.jpg/SmallImage
PriceGuide$$$/PriceGuide
/Product
(product repeats ...)
What's wrong with this code?
I constantly get the error:
The value coldfusion.xml.XmlNodeList cannot be converted to a number.
The error occurred on line 44, which reads:
cfset arrResult = xmlSearch(xmllennox-products,xp)
Suggests, good tutorials? The one I've been following
isn't quite complete enough the complexity of my xml sheet.
Rick
html
head
titleLennox XML Search Test/title
/head
body
h1xmlSearch()/h1
cfset xmlLennox = xmlParse(expandPath('lennox-products.xml'))
cfoutput
table border='1' cellpadding='2' cellspacing='0'
tr
thXPath/th
thResult/th
/tr
cfset arrXPs = arrayNew(1)
cfset arrayAppend(arrXPs,'/products')
cfset arrayAppend(arrXPs,'/products/category')
cfset
arrayAppend(arrXPs,'/products/category/subCategory')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/subCategoryName')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/shortDescription')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/longDescription')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/modelNumber')
cfset
arrayAppend(arrXPs,'/produtts/category/subCategory/product/modelName')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/modelName/text()'
)
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/oneLiner')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/ratings')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/smallImage')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/priceGuide')
cfloop index='index' from='1'
to='#arrayLen(arrXPs)#'
cfset xp = arrXPs[index]
cfset arrResult =
xmlSearch(xmllennox-products,xp)
tr valign='top'
td#xp#/td
td
ul
cfloop
index='index' from='1' to='#arrayLen(arrResult)#'
li#xmlGetNodeType(arrResult[index])#:
cftry
#arrResult[index].xmlText#
cfcatch type='any'#arrResult[index]#/cfcatch
/cftry
/li
/cfloop
/ul
/td
/tr
/cfloop
/table
/cfoutput
/body
/html
-Original Message-
From: James Holmes [mailto:james.hol...@gmail.com]
Sent: Friday, December 30, 2011 8:14 PM
To: cf-talk
Subject: Re: XML! Aaargh! How do I do this?
If you are just after products, you can dramatically simplify the code by
using xmlsearch() to get them with xpath instead of manually looping
multiple levels down.
--
Shu Ha Ri: Agile and .NET blog
http://www.bifrost.com.au/
On 31 December 2011 08:50, Justin Scott leviat...@darktech.org wrote:
Okay... I've worked this every way I can think
of and, I admit it, I just don't get it.
Hi Rick, replace your inner-most loop with the following. It should
get you going in the right direction. It looks like there is another
layer of sub
Re: XML! Aaargh! How do I do this?
Rick, I generally find the easiest way to work through an XML doc is to create a page that just converts it to an XMLDOC first using cfxml or xmlparse() and them CFDUMP the result. This makes it must easier to find the node references you need to loop through or reference. I also prefer to simplify my code by creating a new reference out deep nested blocks, such as nodes I need to loop over, as this results in shorter references, especially if they contains array counters, dynamic evaluations etc to get at the correct element. If your still struggling, then I would suggest posting the full code and XML at pastebin.com so we can see it and test it. -- Russ Michaels www.bluethunderinternet.com : Business hosting services solutions www.cfmldeveloper.com: ColdFusion developer community www.michaels.me.uk : my blog www.cfsearch.com : ColdFusion search engine ** *skype me* : russmichaels ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349285 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: XML! Aaargh! How do I do this?
What are the values of xmllennox in the xml page (or what is it supposed to
be derived from)? And what is the search trying to search for?
-Original Message-
From: Rick Faircloth [mailto:r...@whitestonemedia.com]
Sent: Sunday, January 01, 2012 3:53 PM
To: cf-talk
Subject: RE: XML! Aaargh! How do I do this?
Hopefully, the error of my ways will be obvious to thos of you who work with
xml a lot, but...HELP!!!
Given this demo xml:
?xml version=1.0?
Products
Category ID=1Air Conditioners/Category
SubCategory ID=1
SubCategoryNameAir Conditioners/SubCategoryName
ShortDescription/
LongDescription/
Product ID=XC21 New=N EnergyStar=Y SortOrder=100
ModelNumberXC21/ModelNumber
ModelNameXC21 Air Conditioner/ModelName
OneLinerThe most quiet and efficient central air
conditioner you can buy/OneLiner
Ratings
Rating Type=SEERup to 20.5/Rating
Rating Type=Sound69/Rating
/Ratings
SmallImagehttp://www.lennox.com/res/photos/44383_small.jpg/SmallImage
PriceGuide$$$/PriceGuide
/Product
(product repeats ...)
What's wrong with this code?
I constantly get the error:
The value coldfusion.xml.XmlNodeList cannot be converted to a number.
The error occurred on line 44, which reads:
cfset arrResult = xmlSearch(xmllennox-products,xp)
Suggests, good tutorials? The one I've been following isn't quite complete
enough the complexity of my xml sheet.
Rick
html
head
titleLennox XML Search Test/title
/head
body
h1xmlSearch()/h1
cfset xmlLennox = xmlParse(expandPath('lennox-products.xml'))
cfoutput
table border='1' cellpadding='2' cellspacing='0'
tr
thXPath/th
thResult/th
/tr
cfset arrXPs = arrayNew(1)
cfset arrayAppend(arrXPs,'/products')
cfset arrayAppend(arrXPs,'/products/category')
cfset
arrayAppend(arrXPs,'/products/category/subCategory')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/subCategoryName')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/shortDescription')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/longDescription')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/modelNumber')
cfset
arrayAppend(arrXPs,'/produtts/category/subCategory/product/modelName')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/modelName/text()'
)
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/oneLiner')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/ratings')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/smallImage')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/priceGuide')
cfloop index='index' from='1'
to='#arrayLen(arrXPs)#'
cfset xp = arrXPs[index]
cfset arrResult =
xmlSearch(xmllennox-products,xp)
tr valign='top'
td#xp#/td
td
ul
cfloop
index='index' from='1' to='#arrayLen(arrResult)#'
li#xmlGetNodeType(arrResult[index])#:
cftry
#arrResult[index].xmlText#
cfcatch type='any'#arrResult[index]#/cfcatch
/cftry
/li
/cfloop
/ul
/td
/tr
/cfloop
/table
/cfoutput
/body
/html
-Original Message-
From: James Holmes [mailto:james.hol...@gmail.com]
Sent: Friday, December 30, 2011 8:14 PM
To: cf-talk
Subject: Re: XML! Aaargh! How do I do this?
If you are just after products, you can dramatically simplify the code by
using xmlsearch() to get them with xpath instead of manually looping
multiple levels down.
--
Shu Ha Ri: Agile and .NET blog
http://www.bifrost.com.au
RE: XML! Aaargh! How do I do this?
Sorry, guys... complete forgot to post the CF code!
At this point, all I'm really after is the modelName/text()
(See below)...
I just haven't gotten to the point where I understand how
to parse through the node and info, therein. If you need more
code, I'll go to pastbin.
Thanks for the feedback!
Rick
html
head
titleLennox XML Search Test/title
/head
body
h1xmlSearch()/h1
cfset xmlLennox = xmlParse(expandPath('lennox-products.xml'))
cfoutput
table border='1' cellpadding='2' cellspacing='0'
tr
thXPath/th
thResult/th
/tr
cfset arrXPs = arrayNew(1)
cfset arrayAppend(arrXPs,'/products')
cfset arrayAppend(arrXPs,'/products/category')
cfset
arrayAppend(arrXPs,'/products/category/subCategory')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/subCategoryName')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/shortDescription')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/longDescription')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/modelNumber')
cfset
arrayAppend(arrXPs,'/produtts/category/subCategory/product/modelName')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/modelName/text()'
)
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/oneLiner')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/ratings')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/smallImage')
cfset
arrayAppend(arrXPs,'/products/category/subCategory/product/priceGuide')
cfloop index='index' from='1'
to='#arrayLen(arrXPs)#'
cfset xp = arrXPs[index]
cfset arrResult =
xmlSearch(xmllennox-products,xp)
tr valign='top'
td#xp#/td
td
ul
cfloop
index='index' from='1' to='#arrayLen(arrResult)#'
li#xmlGetNodeType(arrResult[index])#:
cftry
#arrResult[index].xmlText#
cfcatch type='any'#arrResult[index]#/cfcatch
/cftry
/li
/cfloop
/ul
/td
/tr
/cfloop
/table
/cfoutput
/body
/html
-Original Message-
From: Eric Roberts [mailto:ow...@threeravensconsulting.com]
Sent: Sunday, January 01, 2012 6:32 PM
To: cf-talk
Subject: RE: XML! Aaargh! How do I do this?
What are the values of xmllennox in the xml page (or what is it supposed to
be derived from)? And what is the search trying to search for?
-Original Message-
From: Rick Faircloth [mailto:r...@whitestonemedia.com]
Sent: Sunday, January 01, 2012 3:53 PM
To: cf-talk
Subject: RE: XML! Aaargh! How do I do this?
Hopefully, the error of my ways will be obvious to thos of you who work with
xml a lot, but...HELP!!!
Given this demo xml:
?xml version=1.0?
Products
Category ID=1Air Conditioners/Category
SubCategory ID=1
SubCategoryNameAir Conditioners/SubCategoryName
ShortDescription/
LongDescription/
Product ID=XC21 New=N EnergyStar=Y SortOrder=100
ModelNumberXC21/ModelNumber
ModelNameXC21 Air Conditioner/ModelName
OneLinerThe most quiet and efficient central air
conditioner you can buy/OneLiner
Ratings
Rating Type=SEERup to 20.5/Rating
Rating Type=Sound69/Rating
/Ratings
SmallImagehttp://www.lennox.com/res/photos/44383_small.jpg/SmallImage
PriceGuide$$$/PriceGuide
/Product
(product repeats ...)
What's wrong with this code?
I constantly get the error:
The value coldfusion.xml.XmlNodeList cannot be converted to a number.
The error occurred on line 44, which reads:
cfset arrResult = xmlSearch(xmllennox-products,xp)
Suggests, good tutorials? The one I've been following isn't quite complete
enough the complexity of my xml sheet
Re: XML! Aaargh! How do I do this?
Try searching directly for the modelName: cfset result = xmlSearch(xmllennox-products,//modelName) Dump that result to see how to get the name. -- Shu Ha Ri: Agile and .NET blog http://www.bifrost.com.au/ On 2 January 2012 09:18, Rick Faircloth r...@whitestonemedia.com wrote: At this point, all I'm really after is the modelName/text() ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349288 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML! Aaargh! How do I do this?
BTW, the issue with the existing code is this: xmllennox-products CF thinks you are trying to do maths on xmllennox and products by subtracting one from the other. -- Shu Ha Ri: Agile and .NET blog http://www.bifrost.com.au/ On 2 January 2012 10:10, James Holmes james.hol...@gmail.com wrote: xmllennox-products ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349289 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
XML! Aaargh! How do I do this?
Okay... I've worked this every way I can think of and, I admit it, I just don't get it. Where am I going wrong? Ultimately, I want to get the categories of products and list the products of each category. I can get the categories, along with the descriptions, and images, but just can't figure out how to loop through the products. Clues for the cluesless? Rick PS - you can see the cfdumps in the code to show the actual XML I'm trying to work with. I've seen examples of the XML data put into a database and then worked with from there. Would that be preferable? cfscript URLToPull = 'http://lennox.com/api/v1/z7RRSHM/categories/'; /cfscript cfhttp url = '#URLToPull#' method = 'get' timeout = '15' /cfhttp cfscript XMLContent = trim(cfhttp.fileContent); XMLContent = XMLParse(XMLContent); /cfscript cfdump var = #XMLContent# cfoutput cfloop from = '1' to = '#arrayLen(XMLContent.Categories.Category)#' index='idx' #XMLContent.Categories.Category[idx].CategoryName.xmlText#br #XMLContent.Categories.Category[idx].shortDescription.xmlText#br img src='#xmlContent.categories.Category[idx].highlightImage.xmlText#'br br cfscript URLProducts = 'http://lennox.com/api/v1/z7RRSHM/category/#idx#'; /cfscript cfhttp url = '#URLProducts#' method = 'get' timeout = '15' /cfhttp cfscript xmlProductContent = trim(cfhttp.fileContent); xmlProductContent = xmlParse(xmlProductContent); /cfscript cfdump var = #xmlProductContent# cfloop from = '1' to = '#arrayLen(xmlProductContent.products)#' index= 'productCount' #xmlProductContent.products.subcategories[productCount].products.xmlChildren .xmlText#br /cfloop /cfloop /cfoutput The last cfloop above is what I'm having trouble with... Thanks for any help! Rick ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349269 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML! Aaargh! How do I do this?
Okay... I've worked this every way I can think of and, I admit it, I just don't get it. Hi Rick, replace your inner-most loop with the following. It should get you going in the right direction. It looks like there is another layer of sub-categories which needs to be accounted for: !--- Loop over the sub-categories --- cfloop from='1' to='#arrayLen(xmlProductContent.products.SubCategory)#' index='subCategoryCount' !--- Localize our current sub-category --- cfset thisSubCategory = xmlProductContent.products.SubCategory[subCategoryCount] / h2#thisSubCategory.SubCategoryName.xmlText#/h2 !--- Loop over the products in this sub-category --- cfloop from=1 to=#arrayLen(thisSubCategory.Product)# index=productCount !--- Localize our current product for easier access --- cfset thisProduct = thisSubCategory.Product[productCount] / !---cfdump var=#thisProduct# /--- p ID: #thisProduct.xmlAttributes.ID#br / Model Name: #thisProduct.ModelName.xmlText#br / Image: img src=#thisProduct.SmallImage.xmlText# / /p /cfloop !--- Products --- /cfloop !--- Sub-Categories --- -Justin Scott ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349274 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML! Aaargh! How do I do this?
If you are just after products, you can dramatically simplify the code by using xmlsearch() to get them with xpath instead of manually looping multiple levels down. -- Shu Ha Ri: Agile and .NET blog http://www.bifrost.com.au/ On 31 December 2011 08:50, Justin Scott leviat...@darktech.org wrote: Okay... I've worked this every way I can think of and, I admit it, I just don't get it. Hi Rick, replace your inner-most loop with the following. It should get you going in the right direction. It looks like there is another layer of sub-categories which needs to be accounted for: !--- Loop over the sub-categories --- cfloop from='1' to='#arrayLen(xmlProductContent.products.SubCategory)#' index='subCategoryCount' !--- Localize our current sub-category --- cfset thisSubCategory = xmlProductContent.products.SubCategory[subCategoryCount] / h2#thisSubCategory.SubCategoryName.xmlText#/h2 !--- Loop over the products in this sub-category --- cfloop from=1 to=#arrayLen(thisSubCategory.Product)# index=productCount !--- Localize our current product for easier access --- cfset thisProduct = thisSubCategory.Product[productCount] / !---cfdump var=#thisProduct# /--- p ID: #thisProduct.xmlAttributes.ID#br / Model Name: #thisProduct.ModelName.xmlText#br / Image: img src=#thisProduct.SmallImage.xmlText# / /p /cfloop !--- Products --- /cfloop !--- Sub-Categories --- -Justin Scott ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349275 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: XML! Aaargh! How do I do this?
Thanks for the tip, Justin! I'll give it a try! Rick -Original Message- From: Justin Scott [mailto:leviat...@darktech.org] Sent: Friday, December 30, 2011 7:51 PM To: cf-talk Subject: Re: XML! Aaargh! How do I do this? Okay... I've worked this every way I can think of and, I admit it, I just don't get it. Hi Rick, replace your inner-most loop with the following. It should get you going in the right direction. It looks like there is another layer of sub-categories which needs to be accounted for: !--- Loop over the sub-categories --- cfloop from='1' to='#arrayLen(xmlProductContent.products.SubCategory)#' index='subCategoryCount' !--- Localize our current sub-category --- cfset thisSubCategory = xmlProductContent.products.SubCategory[subCategoryCount] / h2#thisSubCategory.SubCategoryName.xmlText#/h2 !--- Loop over the products in this sub-category --- cfloop from=1 to=#arrayLen(thisSubCategory.Product)# index=productCount !--- Localize our current product for easier access --- cfset thisProduct = thisSubCategory.Product[productCount] / !---cfdump var=#thisProduct# /--- p ID: #thisProduct.xmlAttributes.ID#br / Model Name: #thisProduct.ModelName.xmlText#br / Image: img src=#thisProduct.SmallImage.xmlText# / /p /cfloop !--- Products --- /cfloop !--- Sub-Categories --- -Justin Scott ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349276 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: XML! Aaargh! How do I do this?
Thanks for the info, James! I had actually been doing some more research on CF and XML and was reading about xmlSearch. I'll give that try, too. Rick -Original Message- From: James Holmes [mailto:james.hol...@gmail.com] Sent: Friday, December 30, 2011 8:14 PM To: cf-talk Subject: Re: XML! Aaargh! How do I do this? If you are just after products, you can dramatically simplify the code by using xmlsearch() to get them with xpath instead of manually looping multiple levels down. -- Shu Ha Ri: Agile and .NET blog http://www.bifrost.com.au/ On 31 December 2011 08:50, Justin Scott leviat...@darktech.org wrote: Okay... I've worked this every way I can think of and, I admit it, I just don't get it. Hi Rick, replace your inner-most loop with the following. It should get you going in the right direction. It looks like there is another layer of sub-categories which needs to be accounted for: !--- Loop over the sub-categories --- cfloop from='1' to='#arrayLen(xmlProductContent.products.SubCategory)#' index='subCategoryCount' !--- Localize our current sub-category --- cfset thisSubCategory = xmlProductContent.products.SubCategory[subCategoryCount] / h2#thisSubCategory.SubCategoryName.xmlText#/h2 !--- Loop over the products in this sub-category --- cfloop from=1 to=#arrayLen(thisSubCategory.Product)# index=productCount !--- Localize our current product for easier access --- cfset thisProduct = thisSubCategory.Product[productCount] / !---cfdump var=#thisProduct# /--- p ID: #thisProduct.xmlAttributes.ID#br / Model Name: #thisProduct.ModelName.xmlText#br / Image: img src=#thisProduct.SmallImage.xmlText# / /p /cfloop !--- Products --- /cfloop !--- Sub-Categories --- -Justin Scott ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349277 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Save Form Data to XML file
If you don't care about the form of the XML, you can also use WDDX to do the conversion in one quick call. On Wed, Dec 14, 2011 at 10:53 PM, Mike Kear afpwebwo...@gmail.com wrote: IF you use cfdump var=#form# / you should see a struct called form with each of your form fields and their settings from the submitted form. If you use Daniel Gaspar http://www.danielgaspar.com/'s ANYTHINGXOXML.cfc you can automatically convert it to XML without any effort. http://anythingtoxml.riaforge.org/ Cheers Mike Kear Windsor, NSW, Australia Adobe Certified Advanced ColdFusion Developer AFP Webworks http://afpwebworks.com ColdFusion 9 Enterprise, PHP, ASP, ASP.NET hosting from AUD$15/month On Thu, Dec 15, 2011 at 3:43 PM, Tom Jones tjo...@acworld.com wrote: Hello, I have been searching all over cf-talk archives and on google and I don't see a good example on how to save form data to a xml file. So what I'm doing right now is, I have a xml file and read that and then display it in a form so the it can be edited. Here is an example on what I'm thinking. What I'm struggling with is the form data is not returned as an array of structs where I could easily create a xml file from, it's just coma separated lists. Thanks, tom !--- Example XML columns column alignleft/align dnameAgent Version/dname hiddenfalse/hidden idx2/idx nameagent_version/name order2/order width100/width /column column alignleft/align dnameAllow Client/dname hiddenfalse/hidden idx3/idx nameAllowClient/name order1/order width100/width /column /columns --- form name=colSettings method=post table width=600 cellpadding=2 cellspacing=1 tr thOrder/th thDisplay Name/th thAlign Text/th thHide/th thColumn Width/th /tr cfloop index=x array=#sortedColsArray# tr cfoutput tdinput type=text name=order size=3 value=#x.order#/td tdinput type=text name=dname size=50 value=#x.dname#/td tdinput type=text name=align size=10 value=#x.align#/td tdinput type=text name=hidden size=3 value=#x.hidden#/td tdinput type=text name=width size=10 value=#x.width#/td /cfoutput /tr /cfloop /table input name=submit type=submit value=Save / /form ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349162 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Save Form Data to XML file
Thanks, this works great. I really had thought there might have been a simple
function to to do this :-)
thanks again,
tom
So, if you have an arbitrary number of columns and when you submit the
form you get:
form.align=left,left,...
form.dname=Agent Version,Allow Client,...
then you can do:
colCount = listLen(form.align);
xml = ;
for (i = 1; i LTE colCount; i++) {
xml = column;
xml = align#listGetAt(form.align, i)#/align;
xml = dname#listGetAt(form.dname, i)#/dname;
...
xml = /column;
}
xml = columns#xml#/columns;
On 12/14/11 8:43 PM, Tom Jones wrote:
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349166
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Save Form Data to XML file
Hello, I have been searching all over cf-talk archives and on google and I don't see a good example on how to save form data to a xml file. So what I'm doing right now is, I have a xml file and read that and then display it in a form so the it can be edited. Here is an example on what I'm thinking. What I'm struggling with is the form data is not returned as an array of structs where I could easily create a xml file from, it's just coma separated lists. Thanks, tom !--- Example XML columns column alignleft/align dnameAgent Version/dname hiddenfalse/hidden idx2/idx nameagent_version/name order2/order width100/width /column column alignleft/align dnameAllow Client/dname hiddenfalse/hidden idx3/idx nameAllowClient/name order1/order width100/width /column /columns --- form name=colSettings method=post table width=600 cellpadding=2 cellspacing=1 tr thOrder/th thDisplay Name/th thAlign Text/th thHide/th thColumn Width/th /tr cfloop index=x array=#sortedColsArray# tr cfoutput tdinput type=text name=order size=3 value=#x.order#/td tdinput type=text name=dname size=50 value=#x.dname#/td tdinput type=text name=align size=10 value=#x.align#/td tdinput type=text name=hidden size=3 value=#x.hidden#/td tdinput type=text name=width size=10 value=#x.width#/td /cfoutput /tr /cfloop /table input name=submit type=submit value=Save / /form ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349157 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Save Form Data to XML file
So, if you have an arbitrary number of columns and when you submit the
form you get:
form.align=left,left,...
form.dname=Agent Version,Allow Client,...
then you can do:
colCount = listLen(form.align);
xml = ;
for (i = 1; i LTE colCount; i++) {
xml = column;
xml = align#listGetAt(form.align, i)#/align;
xml = dname#listGetAt(form.dname, i)#/dname;
...
xml = /column;
}
xml = columns#xml#/columns;
On 12/14/11 8:43 PM, Tom Jones wrote:
Hello,
I have been searching all over cf-talk archives and on google and I don't see
a good example on how to save form data to a xml file.
So what I'm doing right now is, I have a xml file and read that and then
display it in a form so the it can be edited. Here is an example on what I'm
thinking. What I'm struggling with is the form data is not returned as an
array of structs where I could easily create a xml file from, it's just coma
separated lists.
Thanks,
tom
!--- Example XML
columns
column
alignleft/align
dnameAgent Version/dname
hiddenfalse/hidden
idx2/idx
nameagent_version/name
order2/order
width100/width
/column
column
alignleft/align
dnameAllow Client/dname
hiddenfalse/hidden
idx3/idx
nameAllowClient/name
order1/order
width100/width
/column
/columns
---
form name=colSettings method=post
table width=600 cellpadding=2 cellspacing=1
tr
thOrder/th
thDisplay Name/th
thAlign Text/th
thHide/th
thColumn Width/th
/tr
cfloop index=x array=#sortedColsArray#
tr
cfoutput
tdinput type=text name=order size=3 value=#x.order#/td
tdinput type=text name=dname size=50 value=#x.dname#/td
tdinput type=text name=align size=10 value=#x.align#/td
tdinput type=text name=hidden size=3 value=#x.hidden#/td
tdinput type=text name=width size=10 value=#x.width#/td
/cfoutput
/tr
/cfloop
/table
input name=submit type=submit value=Save /
/form
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349158
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Save Form Data to XML file
IF you use cfdump var=#form# / you should see a struct called form with each of your form fields and their settings from the submitted form. If you use Daniel Gaspar http://www.danielgaspar.com/'s ANYTHINGXOXML.cfc you can automatically convert it to XML without any effort. http://anythingtoxml.riaforge.org/ Cheers Mike Kear Windsor, NSW, Australia Adobe Certified Advanced ColdFusion Developer AFP Webworks http://afpwebworks.com ColdFusion 9 Enterprise, PHP, ASP, ASP.NET hosting from AUD$15/month On Thu, Dec 15, 2011 at 3:43 PM, Tom Jones tjo...@acworld.com wrote: Hello, I have been searching all over cf-talk archives and on google and I don't see a good example on how to save form data to a xml file. So what I'm doing right now is, I have a xml file and read that and then display it in a form so the it can be edited. Here is an example on what I'm thinking. What I'm struggling with is the form data is not returned as an array of structs where I could easily create a xml file from, it's just coma separated lists. Thanks, tom !--- Example XML columns column alignleft/align dnameAgent Version/dname hiddenfalse/hidden idx2/idx nameagent_version/name order2/order width100/width /column column alignleft/align dnameAllow Client/dname hiddenfalse/hidden idx3/idx nameAllowClient/name order1/order width100/width /column /columns --- form name=colSettings method=post table width=600 cellpadding=2 cellspacing=1 tr thOrder/th thDisplay Name/th thAlign Text/th thHide/th thColumn Width/th /tr cfloop index=x array=#sortedColsArray# tr cfoutput tdinput type=text name=order size=3 value=#x.order#/td tdinput type=text name=dname size=50 value=#x.dname#/td tdinput type=text name=align size=10 value=#x.align#/td tdinput type=text name=hidden size=3 value=#x.hidden#/td tdinput type=text name=width size=10 value=#x.width#/td /cfoutput /tr /cfloop /table input name=submit type=submit value=Save / /form ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349159 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: FCKEditor.. XML Request error: Access denied (403)
This is disabled by default, you need to enable it via a Java System Property, see details here: http://www.petefreitag.com/item/718.cfm -- Pete Freitag - Adobe Community Professional http://foundeo.com/ - ColdFusion Consulting Products http://petefreitag.com/ - My Blog http://hackmycf.com - Is your ColdFusion Server Secure? On Wed, Nov 2, 2011 at 7:40 PM, Terry Troxel terry.tro...@gmail.com wrote: Coldfusion 9, just started has been working fine on all the websites until today when using FCKEditor to browse server for an image. Any Ideas? Java recently Updated. Terry ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:348521 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
FCKEditor.. XML Request error: Access denied (403)
Coldfusion 9, just started has been working fine on all the websites until today when using FCKEditor to browse server for an image. Any Ideas? Java recently Updated. Terry ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:348423 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML signature generation failed
No the code does not write to file system. However, it reads a config file and a PFX certificate file to generate the XML signature. Isidro On 10/21/11 9:53 PM, Dave Watts dwa...@figleaf.com wrote: I am invoking a java method that will generate an xml signature. I successfully invoke and define the method, however if the method if the code is run more more than once, the first user gets the XML signature the second user gets the error XML signature generation failed. Can anyone suggest where else other than the application log to see where this is going wrong? Does the code write anything to the filesystem? If so, maybe you're running into a problem overwriting the file created on the first iteration. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:348317 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
XML signature generation failed
Hello all, I am invoking a java method that will generate an xml signature. I successfully invoke and define the method, however if the method if the code is run more more than once, the first user gets the XML signature the second user gets the error XML signature generation failed. Can anyone suggest where else other than the application log to see where this is going wrong? cfscript my = CreateObject(java, com.connect.integration.Provider); my.init(); ret = my.sendResponse(getPageContext().getServletContext(),getPageContext().getResponse(),urn:oasis:names:tc:SAML:2.0:ac:classes:Password.getAuthnContext,JavaCast(string,#SESSION.myuid#),#myStruct#); /cfscript Thank You in advance for any help. Isidro ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:348299 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML signature generation failed
I am invoking a java method that will generate an xml signature. I successfully invoke and define the method, however if the method if the code is run more more than once, the first user gets the XML signature the second user gets the error XML signature generation failed. Can anyone suggest where else other than the application log to see where this is going wrong? Does the code write anything to the filesystem? If so, maybe you're running into a problem overwriting the file created on the first iteration. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:348301 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
XML parse
Hi all I tried to parse the XML from the following feed as I normally do and kept getting error messages (not defined) on everything I tried. http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682 Here are a few snippets that I tried: cfset xmldoc = XmlParse(http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682;) cfloop list=#xmldoc# index=i cfoutput #xmlDoc.jobs.job.title.XmlText#br / /cfoutput /cfloop cfset myxmldoc= XmlParse(http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682;) cfset title = XmlSearch(myxmldoc, /jobs/job/title) cfset location= XmlSearch(myxmldoc, /jobs/job/location) cfset department = XmlSearch(myxmldoc, /jobs/job/department) cfset referencenumber = XmlSearch(myxmldoc, /jobs/job/referencenumber) cfset link= XmlSearch(myxmldoc, /jobs/job/url) cfset description = XmlSearch(myxmldoc, /jobs/job/description) Any suggestions? ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:347555 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML parse
Jobs is capitalized in the XML. This works: cfset myxmldoc = XmlParse( http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682;) cfset title = XmlSearch(myxmldoc, /Jobs/job/title) cfdump var=#title# On Wed, Sep 21, 2011 at 9:02 AM, Torrent Girl moniqueb...@gmail.com wrote: Hi all I tried to parse the XML from the following feed as I normally do and kept getting error messages (not defined) on everything I tried. http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682 Here are a few snippets that I tried: cfset xmldoc = XmlParse( http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682;) cfloop list=#xmldoc# index=i cfoutput #xmlDoc.jobs.job.title.XmlText#br / /cfoutput /cfloop cfset myxmldoc= XmlParse( http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682;) cfset title = XmlSearch(myxmldoc, /jobs/job/title) cfset location= XmlSearch(myxmldoc, /jobs/job/location) cfset department = XmlSearch(myxmldoc, /jobs/job/department) cfset referencenumber = XmlSearch(myxmldoc, /jobs/job/referencenumber) cfset link= XmlSearch(myxmldoc, /jobs/job/url) cfset description = XmlSearch(myxmldoc, /jobs/job/description) Any suggestions? ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:347556 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: XML parse
are you sure that CF is actually getting the XML ? try grabbing it with CFHTTP first instead just to make sure. On Wed, Sep 21, 2011 at 2:02 PM, Torrent Girl moniqueb...@gmail.com wrote: Hi all I tried to parse the XML from the following feed as I normally do and kept getting error messages (not defined) on everything I tried. http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682 Here are a few snippets that I tried: cfset xmldoc = XmlParse(http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682;) cfloop list=#xmldoc# index=i cfoutput #xmlDoc.jobs.job.title.XmlText#br / /cfoutput /cfloop cfset myxmldoc = XmlParse(http://feeds.jobs2web.com/feeds/view?siteId=246feedId=682;) cfset title = XmlSearch(myxmldoc, /jobs/job/title) cfset location = XmlSearch(myxmldoc, /jobs/job/location) cfset department = XmlSearch(myxmldoc, /jobs/job/department) cfset referencenumber = XmlSearch(myxmldoc, /jobs/job/referencenumber) cfset link = XmlSearch(myxmldoc, /jobs/job/url) cfset description = XmlSearch(myxmldoc, /jobs/job/description) Any suggestions? ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:347557 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Spam:*******, Re: RETS - Real Estate Standards Organization - XML feed - anyone have any experience
Hi Wil, Thanks for the note.. A couple of questions When you mention use phRets you're suggesting using a php server to accomplish the data retrieval tasks? With regard to the amount of data that is retrieved each time we connect to the RETS server... currently I pull information via an FTP feed. I connect to the their FTP site and look for the latest ftp zip file. These zip files allow me to add and delete properties from my aggregate database. Is there some sort of query language with RETS that allows me to do the same incremental update throughout the day? Finally, I run my own CF server... allow I'm no expert on incorporating a JAVA API into CF I know that it is possible. Have you looked at RETS IQ Java RETS Library? http://wiki.rets.org/wiki/RETS_IQ_Java_RETS_Library What do you think about using such a library? Thanks again! Wayne On 8/6/2011 12:50 PM, Wil Genovese wrote: Wayne, I spent over 5 years working with Real Estate data and the RETS servers. The misfortunate part of RETS is that there is NOT a standard. Every MLS market implements RETS differently. I've written a RETS client in ColdFusion once. It's market specific and the client owns the code. The best I can suggest is to look at phRets and decided if you want to use that or rewrite that to ColdFusion. The thing to remember it that it's all XML. NExt you have to make http calls to get the meta data, then parse that out then do another http call to get the next level meta data and parse that out. You keep doing this until you have all the meta data that describes the entire data structure. Then create the structure in your DB. Then you start doing http calls to get the data. If the market is a large MLS market, you may have problems with memory limits in the JVM. I've had to run upwards of 4Gb or more in the heap and setup fairly aggressive garbage collections. Your server side needs to be fast. The process of RETS means the code will be running for maybe an hour or more (depending on the data set) and could be making hundreds of HTTP calls. Getting images is almost as fun. But I found I could get them with ColdFusion via RETS and save them to a drive rather quickly. The data is in binary format and may not seem obvious at first when you make the first RETS object call. But ColdFusion will process the data and you can save the image. I just looked in my junk code folder where I keep all sorts of test code snippets. I found some RETS test code that may help you. I'll send it privately. Good Luck! Wil Genovese Sr. Web Application Developer/ Systems Administrator CF Webtools www.cfwebtools.com wilg...@trunkful.com www.trunkful.com On Aug 6, 2011, at 2:36 PM, Wayne Gregor wrote: I need to convert over a couple of real estate website that are currently receiving MLS data via an hourly FTP feed. The new system handed down by the National Association of Realtors is RETShttp://www.rets.org/. Do any of you know if there are any CFC functions out there that would make it easier for me to interface with this new system? -- *Wayne Gregori* Office: 510-895-1066 Cell: 510-219-3887 w...@sfnet.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:346721 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RETS - Real Estate Standards Organization - XML feed - anyone have any experience
I need to convert over a couple of real estate website that are currently receiving MLS data via an hourly FTP feed. The new system handed down by the National Association of Realtors is RETS http://www.rets.org/. Do any of you know if there are any CFC functions out there that would make it easier for me to interface with this new system? -- *Wayne Gregori* Office: 510-895-1066 Cell: 510-219-3887 w...@sfnet.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:346559 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: RETS - Real Estate Standards Organization - XML feed - anyone have any experience
Wayne, I spent over 5 years working with Real Estate data and the RETS servers. The misfortunate part of RETS is that there is NOT a standard. Every MLS market implements RETS differently. I've written a RETS client in ColdFusion once. It's market specific and the client owns the code. The best I can suggest is to look at phRets and decided if you want to use that or rewrite that to ColdFusion. The thing to remember it that it's all XML. NExt you have to make http calls to get the meta data, then parse that out then do another http call to get the next level meta data and parse that out. You keep doing this until you have all the meta data that describes the entire data structure. Then create the structure in your DB. Then you start doing http calls to get the data. If the market is a large MLS market, you may have problems with memory limits in the JVM. I've had to run upwards of 4Gb or more in the heap and setup fairly aggressive garbage collections. Your server side needs to be fast. The process of RETS means the code will be running for maybe an hour or more (depending on the data set) and could be making hundreds of HTTP calls. Getting images is almost as fun. But I found I could get them with ColdFusion via RETS and save them to a drive rather quickly. The data is in binary format and may not seem obvious at first when you make the first RETS object call. But ColdFusion will process the data and you can save the image. I just looked in my junk code folder where I keep all sorts of test code snippets. I found some RETS test code that may help you. I'll send it privately. Good Luck! Wil Genovese Sr. Web Application Developer/ Systems Administrator CF Webtools www.cfwebtools.com wilg...@trunkful.com www.trunkful.com On Aug 6, 2011, at 2:36 PM, Wayne Gregor wrote: I need to convert over a couple of real estate website that are currently receiving MLS data via an hourly FTP feed. The new system handed down by the National Association of Realtors is RETS http://www.rets.org/. Do any of you know if there are any CFC functions out there that would make it easier for me to interface with this new system? -- *Wayne Gregori* Office: 510-895-1066 Cell: 510-219-3887 w...@sfnet.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:346561 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Coldfusion 9 XML creation and HTML after
Here is what I am running into. I have a web form that takes in the info I need. I process the form on the next page and it creates an XML document that I need (which is being created successfully). I need to display something back to the end user telling them it's been created. After my end tag /cfprocessingdirective if I write any text it throws back the errors that there is something after that end tag... How do I create my XML document and still display something back to the end user (or if need be, use a cflocation to get them to an acknowledgement page? The error that comes back in a red box: This page contains the following errors: error on line 26 at column 1: Extra content at the end of the document Below is a rendering of the page up to the first error. Ken ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344499 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Coldfusion 9 XML creation and HTML after
can you post the code here http://pastebin.com/ http://pastebin.com/ On Fri, May 13, 2011 at 5:43 PM, Ken Hammond khamm...@saleminc.com wrote: Here is what I am running into. I have a web form that takes in the info I need. I process the form on the next page and it creates an XML document that I need (which is being created successfully). I need to display something back to the end user telling them it's been created. After my end tag /cfprocessingdirective if I write any text it throws back the errors that there is something after that end tag... How do I create my XML document and still display something back to the end user (or if need be, use a cflocation to get them to an acknowledgement page? The error that comes back in a red box: This page contains the following errors: error on line 26 at column 1: Extra content at the end of the document Below is a rendering of the page up to the first error. Ken ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344502 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Coldfusion 9 XML creation and HTML after
Here is what I am running into. I have a web form that takes in the info I need. I process the form on the next page and it creates an XML document that I need (which is being created successfully). I need to display something back to the end user telling them it's been created. After my end tag /cfprocessingdirective if I write any text it throws back the errors that there is something after that end tag... How do I create my XML document and still display something back to the end user (or if need be, use a cflocation to get them to an acknowledgement page? You can't output both XML and HTML within a single HTTP response. Well, you can, but the client won't know what to do with it. If you need to generate XML, that typically isn't going to be returned directly to the browser, because the browser won't know what to do with it. So, you will have to take a slightly different approach. There are a couple of ways you could go. You could notify the user before generating the XML, then redirect the user to the XML generation. Or, you could generate the XML and store it in a variable or file, then allow the user to click a link to generate the appropriate XML response. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or on ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344503 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Coldfusion 9 XML creation and HTML after
my guess is that you have answered your question by saying that you could use a cflocation tag. Alternatively, if you trust your xml scrip to always create the xml document without errors, you may use a prompt to display your activity status just before you use the xml script. Best Regrads, Chuka I.W. Anene Chief Software Eng./CEO Quorium Solutions www.quorium.org +234 70 3269 6113 +234 80 9530 7257 +234 1 881 2777 From: Ken Hammond khamm...@saleminc.com To: cf-talk cf-talk@houseoffusion.com Sent: Fri, May 13, 2011 4:43:44 PM Subject: Coldfusion 9 XML creation and HTML after Here is what I am running into. I have a web form that takes in the info I need. I process the form on the next page and it creates an XML document that I need (which is being created successfully). I need to display something back to the end user telling them it's been created. After my end tag /cfprocessingdirective if I write any text it throws back the errors that there is something after that end tag... How do I create my XML document and still display something back to the end user (or if need be, use a cflocation to get them to an acknowledgement page? The error that comes back in a red box: This page contains the following errors: error on line 26 at column 1: Extra content at the end of the document Below is a rendering of the page up to the first error. Ken ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344505 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
Hi all. I have been able to output my values from the xml (YAY!!) using xmlSearch (Thanks Guys). Here is my code: cfscript myxmldoc = XmlParse(E:\domains\aawit.net\wwwroot\Microsoft-Diversity-AAWIT116obs.xml); title= XmlSearch(myxmldoc, /jobs/job/title); location = XmlSearch(myxmldoc, /jobs/job/location); department = XmlSearch(myxmldoc, /jobs/job/department); referencenumber = XmlSearch(myxmldoc, /jobs/job/referencenumber); description = XmlSearch(myxmldoc, /jobs/job/description); for (i = 1; i LTE ArrayLen(title); i = i + 1) writeoutput(trtd title[i].XmlText /tdtd location[i].XmlText /tdtd department[i].XmlText /tdtd referencenumber[i].XmlText /tdtd description[i].XmlText /td/trtrtd description[i].XmlText /td/tr); /cfscript What I'd like to do is to be able to add these values to a table in my database but when I try to take the above code out of CFSCRIPT and do a straight output, it doesn't work. Any ideas on how I can do this? TIA ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344302 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
If you're using SQL Server, it'll auto-handle the XML directly with no pre-parsing with CF (SEE http://msdn.microsoft.com/en-us/library/ms186918.aspx ): cfstoredproc procedure=BulkInsert cfprocparam type=In cfsqltype=CF_SQL_LONGVARCHAR value=#varContainingXML# /cfstoredproc PROCEDURE [BulkInsert] @doc text AS BEGIN DECLARE @idoc int --Create an internal representation of the XML document. EXEC sp_xml_preparedocument @idoc OUTPUT, @doc INSERT INTO yourTable (colum1, colum2, ...) SELECT xmlnodename1, xmlnodename2, ... FROM OPENXML(@idoc, '/xml/path/to/rows', 0) WITH ( xmlnodename1 datatype, xmlnodename2 datatype, ...) EXEC sp_xml_removedocument @idoc END On Fri, May 6, 2011 at 2:46 PM, Torrent Girl moniqueb...@gmail.com wrote: Hi all. I have been able to output my values from the xml (YAY!!) using xmlSearch (Thanks Guys). Here is my code: cfscript myxmldoc = XmlParse(E:\domains\aawit.net \wwwroot\Microsoft-Diversity-AAWIT116obs.xml); title= XmlSearch(myxmldoc, /jobs/job/title); location = XmlSearch(myxmldoc, /jobs/job/location); department = XmlSearch(myxmldoc, /jobs/job/department); referencenumber = XmlSearch(myxmldoc, /jobs/job/referencenumber); description = XmlSearch(myxmldoc, /jobs/job/description); for (i = 1; i LTE ArrayLen(title); i = i + 1) writeoutput(trtd title[i].XmlText /tdtd location[i].XmlText /tdtd department[i].XmlText /tdtd referencenumber[i].XmlText /tdtd description[i].XmlText /td/trtrtd description[i].XmlText /td/tr); /cfscript What I'd like to do is to be able to add these values to a table in my database but when I try to take the above code out of CFSCRIPT and do a straight output, it doesn't work. Any ideas on how I can do this? TIA ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344303 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
stick it inside a cfsavecontent -- Russ Michaels www.bluethunderinternet.com : Business hosting services solutions www.cfmldeveloper.com: ColdFusion developer community www.michaels.me.uk : my blog www.cfsearch.com : ColdFusion search engine ** *skype me* : russmichaels ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344304 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Coldfusion 9 SQL Server 2008 XML bulkload
First SQL has to be running under an account that has access to that directory.
Then it is easiest if you make a share, then you reference it as
\\sernername\sharename\filename
Steve
Sent from my iPhone
On May 3, 2011, at 12:48 AM, Kam Heydari heyda...@yahoo.com wrote:
Hi,
Anyone know how to pass file name to the bulkload when sqlserver and
coldfision servers are on different physical servers; The following xml load
works when both SQl and CF servers are residing on the same physical server.
- How I can refer to the local file where sqlserver can access it?
- Any alternative ways?
Thanks
K
EXEC('
INSERT INTO [ProjectXML](Code, Reference, xmlFileName, [xmlDocument])
SELECT '''+'2503'+''','''+'455'+''', ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''', xmlData FROM ( SELECT * FROM OPENROWSET (BULK ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''' , SINGLE_BLOB) AS XMLDATA ) AS FileImport (XMLDATA) ')
Error when on different servers:
Message: Error Executing Database Query.
Detail: [Macromedia][SQLServer JDBC Driver][SQLServer]Cannot bulk load
because the file
C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp4527544419188947137.tmp
could not be opened. Operating system error code 3(The system cannot find
the path specified.).
sqlStatement: HY000
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344155
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Coldfusion 9 SQL Server 2008 XML bulkload
Or you should be able to read the file with cffile and then pass it to SQL
Server in a cfprocparam...
On Tue, May 3, 2011 at 6:01 AM, Steven Durette st...@durette.org wrote:
First SQL has to be running under an account that has access to that
directory. Then it is easiest if you make a share, then you reference it as
\\sernername\sharename\filename
Steve
Sent from my iPhone
On May 3, 2011, at 12:48 AM, Kam Heydari heyda...@yahoo.com wrote:
Hi,
Anyone know how to pass file name to the bulkload when sqlserver and
coldfision servers are on different physical servers; The following xml load
works when both SQl and CF servers are residing on the same physical server.
- How I can refer to the local file where sqlserver can access it?
- Any alternative ways?
Thanks
K
EXEC('
INSERT INTO [ProjectXML](Code, Reference, xmlFileName, [xmlDocument])
SELECT '''+'2503'+''','''+'455'+''', ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''', xmlData FROM ( SELECT * FROM OPENROWSET (BULK ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''' , SINGLE_BLOB) AS XMLDATA ) AS FileImport (XMLDATA) ')
Error when on different servers:
Message: Error Executing Database Query.
Detail: [Macromedia][SQLServer JDBC Driver][SQLServer]Cannot bulk load
because the file
C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp4527544419188947137.tmp
could not be opened. Operating system error code 3(The system cannot find
the path specified.).
sqlStatement: HY000
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344156
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Coldfusion 9 SQL Server 2008 XML bulkload
Thanks Steve,
Kambiz Heydari
(781) 898-8857
On May 3, 2011, at 7:01 AM, Steven Durette st...@durette.org wrote:
First SQL has to be running under an account that has access to that
directory. Then it is easiest if you make a share, then you reference it as
\\sernername\sharename\filename
Steve
Sent from my iPhone
On May 3, 2011, at 12:48 AM, Kam Heydari heyda...@yahoo.com wrote:
Hi,
Anyone know how to pass file name to the bulkload when sqlserver and
coldfision servers are on different physical servers; The following xml load
works when both SQl and CF servers are residing on the same physical server.
- How I can refer to the local file where sqlserver can access it?
- Any alternative ways?
Thanks
K
EXEC('
INSERT INTO [ProjectXML](Code, Reference, xmlFileName, [xmlDocument])
SELECT '''+'2503'+''','''+'455'+''', ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''', xmlData FROM ( SELECT * FROM OPENROWSET (BULK ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''' , SINGLE_BLOB) AS XMLDATA ) AS FileImport (XMLDATA) ')
Error when on different servers:
Message: Error Executing Database Query.
Detail: [Macromedia][SQLServer JDBC Driver][SQLServer]Cannot bulk load
because the file
C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp4527544419188947137.tmp
could not be opened. Operating system error code 3(The system cannot find
the path specified.).
sqlStatement: HY000
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344157
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
MS XML Parse
Hello all. I am trying to parse thru an XML feed using the method that I always use and it's not working: Here is my code: cfset getJobs = 'http://www.microsoft-careers.com/Microsoft-Diversity-AAWIT116obs.xml' cfhttp url=#getJobs# method=GET timeout=1/cfhttp !--- Read the file and convert it to an XML document object--- cfset msJobs = XmlParse(cfhttp.filecontent) cfdump var=#msJobs# The dump doesn't show up. I just get a blank screen. Any suggestions? TIA ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344161 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: MS XML Parse
Are you sure that someone didn't screw up the XML in the file? -Original Message- From: Torrent Girl [mailto:moniqueb...@gmail.com] Sent: Tuesday, May 03, 2011 12:15 PM To: cf-talk Subject: MS XML Parse Hello all. I am trying to parse thru an XML feed using the method that I always use and it's not working: Here is my code: cfset getJobs = 'http://www.microsoft-careers.com/Microsoft-Diversity-AAWIT116obs.xml' cfhttp url=#getJobs# method=GET timeout=1/cfhttp !--- Read the file and convert it to an XML document object--- cfset msJobs = XmlParse(cfhttp.filecontent) cfdump var=#msJobs# The dump doesn't show up. I just get a blank screen. Any suggestions? TIA ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344162 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
Maybe it's too big a file this time. the XMlParse function isn't the best when it gets to large documents, time to drop into java :) SAX parsers work even better for large documents. I looked at it and it seems to be a rather big file BTW. Regards Mark Drew On 3 May 2011, at 12:14, Torrent Girl wrote: Hello all. I am trying to parse thru an XML feed using the method that I always use and it's not working: Here is my code: cfset getJobs = 'http://www.microsoft-careers.com/Microsoft-Diversity-AAWIT116obs.xml' cfhttp url=#getJobs# method=GET timeout=1/cfhttp !--- Read the file and convert it to an XML document object--- cfset msJobs = XmlParse(cfhttp.filecontent) cfdump var=#msJobs# The dump doesn't show up. I just get a blank screen. Any suggestions? TIA ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344165 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
On 5/3/2011 9:14 AM, Torrent Girl wrote: cfset getJobs = 'http://www.microsoft-careers.com/Microsoft-Diversity-AAWIT116obs.xml' That looks to be a pretty large XML document. There are definite limits on how large of document that xmlParse() can process. As xmlParse uses a DOM parsing methodology, the entire resulting data structure (which will be several times larger the the original text document) must fit into memory. If this document is too large to dom parse you need to do something that parses it segment by segment, such as a sax parser. ColdFusion does not have a built in sax parser. I used Ben Nadel's pseudo parser [http://www.bennadel.com/blog/1345-Ask-Ben-Parsing-Very-Large-XML-Documents-In-ColdFusion.htm] once when I needed a quick solution. One could also dive into the underlining Java and access some true sax parser through it. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344167 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
On 5/3/2011 9:14 AM, Torrent Girl wrote: cfset getJobs = 'http://www.microsoft-careers. com/Microsoft-Diversity-AAWIT116obs.xml' That looks to be a pretty large XML document. There are definite limits on how large of document that xmlParse() can process. As xmlParse uses a DOM parsing methodology, the entire resulting data structure (which will be several times larger the the original text document) must fit into memory. If this document is too large to dom parse you need to do something that parses it segment by segment, such as a sax parser. ColdFusion does not have a built in sax parser. I used Ben Nadel's pseudo parser [http://www.bennadel. com/blog/1345-Ask-Ben-Parsing-Very-Large-XML-Documents-In-ColdFusion. htm] once when I needed a quick solution. One could also dive into the underlining Java and access some true sax parser through it. Actually, I can dump the contents of the cfhttp.filecontent return with no problem. cfset getJobs = 'http://www.microsoft-careers.com/Microsoft-Diversity-AAWIT116obs.xml' cfhttp url=#getJobs# method=GET timeout=1/cfhttp cfdump var=#cfhttp.filecontent# top=10 I usually have to dump the parse to see what available for outputing the actual values. Here is my page with the cfhttp content dumped: http://www.aawit.net/xmlJobFeedtest.cfm If I can get the structure of the values I'd be good: i.e., #msjobs.job.title.XmlText# I am not familiar with XML enough to know how to output the parsing. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344172 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
On 5/3/2011 10:18 AM, Torrent Girl wrote: Actually, I can dump the contents of the cfhttp.filecontent return with no problem. That is because at that point, the data is still flat text. One can fit a WHOLE LOT of flat text into a gigabyte or two of ram memory. The problem will occur when you try to parse that flat text into a DOM relational data structure. Now IF you do not need to parse the flat data, you can use xmlTransform(), and maybe xmlSearch(), to apply XPATH to the large flat text data and create smaller subsets of the XML. That might get you where you need to go. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344176 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: MS XML Parse
On 5/3/2011 10:18 AM, Torrent Girl wrote: Actually, I can dump the contents of the cfhttp.filecontent return with no problem. That is because at that point, the data is still flat text. One can fit a WHOLE LOT of flat text into a gigabyte or two of ram memory. The problem will occur when you try to parse that flat text into a DOM relational data structure. Now IF you do not need to parse the flat data, you can use xmlTransform(), and maybe xmlSearch(), to apply XPATH to the large flat text data and create smaller subsets of the XML. That might get you where you need to go. Great thanks! I'll try one of these. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344179 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Coldfusion 9 SQL Server 2008 XML bulkload
Hi,
Anyone know how to pass file name to the bulkload when sqlserver and coldfision
servers are on different physical servers; The following xml load works when
both SQl and CF servers are residing on the same physical server.
- How I can refer to the local file where sqlserver can access it?
- Any alternative ways?
Thanks
K
EXEC('
INSERT INTO [ProjectXML](Code, Reference, xmlFileName, [xmlDocument])
SELECT '''+'2503'+''','''+'455'+''', ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''', xmlData FROM ( SELECT * FROM OPENROWSET (BULK ''' +
'C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp6410035959573708012.tmp'
+ ''' , SINGLE_BLOB) AS XMLDATA ) AS FileImport (XMLDATA) ')
Error when on different servers:
Message: Error Executing Database Query.
Detail: [Macromedia][SQLServer JDBC Driver][SQLServer]Cannot bulk load because
the file
C:\ColdFusion9\runtime\servers\coldfusion\SERVER-INF\temp\wwwroot-tmp\neotmp4527544419188947137.tmp
could not be opened. Operating system error code 3(The system cannot find the
path specified.).
sqlStatement: HY000
.
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344153
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Here's the update: Jochem's StAX-based solution worked so well -- and it allowed me to re-use existing code since I was already processing xml -- that I didn't bother to consider testing the xml2csv utility. Original code processing a 45mb file yielded an insert time of 90 seconds per record with total file processing time being a projected 66 hours. Using code that incorporated xmlSplitter.cfc, the routine created 2682 discrete xml files in approximately 45 seconds. From there, the insert loop did its work pulling out xml and pouring it into roughly 100 db table fields at ... 20 records per second. Total time to read in the 45mb xml file, create the discrete files, read them, insert them into the db and delete them (one at a time as I went along in the loop) was 192250ms. A bit more than three minutes. Thats an acceptable level of improvement. I owe you one, Jochem. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342450 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Nice report! I'll try and remember that the next time I have to parse huge XML files. :-) ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342451 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
Nice... -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Saturday, February 19, 2011 9:06 PM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? Here's the update: Jochem's StAX-based solution worked so well -- and it allowed me to re-use existing code since I was already processing xml -- that I didn't bother to consider testing the xml2csv utility. Original code processing a 45mb file yielded an insert time of 90 seconds per record with total file processing time being a projected 66 hours. Using code that incorporated xmlSplitter.cfc, the routine created 2682 discrete xml files in approximately 45 seconds. From there, the insert loop did its work pulling out xml and pouring it into roughly 100 db table fields at ... 20 records per second. Total time to read in the 45mb xml file, create the discrete files, read them, insert them into the db and delete them (one at a time as I went along in the loop) was 192250ms. A bit more than three minutes. Thats an acceptable level of improvement. I owe you one, Jochem. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342452 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
Matt, Very cool - an innovative approach I would not have thought of. Of course we all know that Jochem's a certified genius :) -Mark Mark A. Kruger, MCSE, CFG (402) 408-3733 ext 105 Skype: markakruger www.cfwebtools.com www.coldfusionmuse.com www.necfug.com -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Saturday, February 19, 2011 8:06 PM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? Here's the update: Jochem's StAX-based solution worked so well -- and it allowed me to re-use existing code since I was already processing xml -- that I didn't bother to consider testing the xml2csv utility. Original code processing a 45mb file yielded an insert time of 90 seconds per record with total file processing time being a projected 66 hours. Using code that incorporated xmlSplitter.cfc, the routine created 2682 discrete xml files in approximately 45 seconds. From there, the insert loop did its work pulling out xml and pouring it into roughly 100 db table fields at ... 20 records per second. Total time to read in the 45mb xml file, create the discrete files, read them, insert them into the db and delete them (one at a time as I went along in the loop) was 192250ms. A bit more than three minutes. Thats an acceptable level of improvement. I owe you one, Jochem. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342453 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Wed, Feb 16, 2011 at 8:08 PM, Rick Faircloth wrote: Perhaps this will be of some help. Thanks Rick, Interesting stuff. I don't work with xml so much so a lot of this is new, and I hate new :-). I haven't been a full-time programmer for a few years now so I haven't kept up with everything like I would have otherwise. I have a business trip today so I have to set this aside but I'm going to run Jochem's solution - which is presented darn near turnkey - as soon as I can tomorrow and see what happens. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342379 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
No problem... I haven't done much with XML, myself, so I thought this presented a good opportunity to learn something new, too. Let us know how the solution you're working on pans out. -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Thursday, February 17, 2011 9:42 AM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? On Wed, Feb 16, 2011 at 8:08 PM, Rick Faircloth wrote: Perhaps this will be of some help. Thanks Rick, Interesting stuff. I don't work with xml so much so a lot of this is new, and I hate new :-). I haven't been a full-time programmer for a few years now so I haven't kept up with everything like I would have otherwise. I have a business trip today so I have to set this aside but I'm going to run Jochem's solution - which is presented darn near turnkey - as soon as I can tomorrow and see what happens. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342381 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
In our company I found a solution to parse big XML files in ColdFusion applications: 1) using xml2csv(http://www.a7soft.com/xml2csv.html), a very fast application to convert XML to csv file; 2) with SQL Server BULK INSERT to insert in our database Using this approach reduced time in 100 times. Very fast. Cheers Marco Antonio On Thu, Feb 17, 2011 at 1:30 PM, Rick Faircloth r...@whitestonemedia.comwrote: No problem... I haven't done much with XML, myself, so I thought this presented a good opportunity to learn something new, too. Let us know how the solution you're working on pans out. -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Thursday, February 17, 2011 9:42 AM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? On Wed, Feb 16, 2011 at 8:08 PM, Rick Faircloth wrote: Perhaps this will be of some help. Thanks Rick, Interesting stuff. I don't work with xml so much so a lot of this is new, and I hate new :-). I haven't been a full-time programmer for a few years now so I haven't kept up with everything like I would have otherwise. I have a business trip today so I have to set this aside but I'm going to run Jochem's solution - which is presented darn near turnkey - as soon as I can tomorrow and see what happens. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342382 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
WE've used this approach as well... seems like a kludge but it actually works really really well. MS Text driver and DTS (or SSIS) are exceptionally fast for importing textual data. We have a process that consumes 350,000 rows of data in about 5-8 seconds... and does it every 2 minutes. -Mark -Original Message- From: Marco Antonio C. Santos [mailto:marcoacsan...@gmail.com] Sent: Thursday, February 17, 2011 9:54 AM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? In our company I found a solution to parse big XML files in ColdFusion applications: 1) using xml2csv(http://www.a7soft.com/xml2csv.html), a very fast application to convert XML to csv file; 2) with SQL Server BULK INSERT to insert in our database Using this approach reduced time in 100 times. Very fast. Cheers Marco Antonio On Thu, Feb 17, 2011 at 1:30 PM, Rick Faircloth r...@whitestonemedia.comwrote: No problem... I haven't done much with XML, myself, so I thought this presented a good opportunity to learn something new, too. Let us know how the solution you're working on pans out. -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Thursday, February 17, 2011 9:42 AM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? On Wed, Feb 16, 2011 at 8:08 PM, Rick Faircloth wrote: Perhaps this will be of some help. Thanks Rick, Interesting stuff. I don't work with xml so much so a lot of this is new, and I hate new :-). I haven't been a full-time programmer for a few years now so I haven't kept up with everything like I would have otherwise. I have a business trip today so I have to set this aside but I'm going to run Jochem's solution - which is presented darn near turnkey - as soon as I can tomorrow and see what happens. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342383 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
Good to know! -Original Message- From: Marco Antonio C. Santos [mailto:marcoacsan...@gmail.com] Sent: Thursday, February 17, 2011 10:54 AM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? In our company I found a solution to parse big XML files in ColdFusion applications: 1) using xml2csv(http://www.a7soft.com/xml2csv.html), a very fast application to convert XML to csv file; 2) with SQL Server BULK INSERT to insert in our database Using this approach reduced time in 100 times. Very fast. Cheers Marco Antonio On Thu, Feb 17, 2011 at 1:30 PM, Rick Faircloth r...@whitestonemedia.comwrote: No problem... I haven't done much with XML, myself, so I thought this presented a good opportunity to learn something new, too. Let us know how the solution you're working on pans out. -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Thursday, February 17, 2011 9:42 AM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? On Wed, Feb 16, 2011 at 8:08 PM, Rick Faircloth wrote: Perhaps this will be of some help. Thanks Rick, Interesting stuff. I don't work with xml so much so a lot of this is new, and I hate new :-). I haven't been a full-time programmer for a few years now so I haven't kept up with everything like I would have otherwise. I have a business trip today so I have to set this aside but I'm going to run Jochem's solution - which is presented darn near turnkey - as soon as I can tomorrow and see what happens. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342388 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Thu, Feb 17, 2011 at 8:56 AM, Rick Faircloth wrote: Good to know! Indeed. Thanks to Marco and Mark. I plan on trying out both this and Jochem's solution starting maybe as soon as tomorrow and certainly through the long U.S. holiday weekend. Maybe try both and see which is faster. I'll post back the results. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342417 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Hey everybody, we're in a bind here and I pitched the idea to the client of bringing in a ringer. Is there someone out there who is interested in building a direct-to-SQL Server or direct-to-mySQL-based solution? Solution would have to, on a scheduled basis, grab a file in a location on the server on its own without CF and process it (we have stuff that retrieves/places the file). Current file size is about 50 mb but must be expected to go to around 100. At 42mb using just CF I am at about 70 seconds per record loop. We need a *significant* improvement in performance. Work would be on a dedicated Windows server. Present box is a fairly powerful Crystaltech Win2003 Server w/4GB RAM and a Xeon 2.8 processor w/4 cores. We'd consider a Win2008, 64-bit installation to improve speed. Looking at a 24-core server at Viviotech w/16gb as a hi-horsepower alternative. Our feed partner is probably going to be able to use a different method of delivery that will reduce file size, but for now we need to plan for the worst and move on it. Anyone interested? You can email me at my for-reals email at matt AT mysecretbase DOT com. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342336 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
Hi, Matt... I know you might have looked at this link at dev.mysql.com for a solution to processing your data feed, but I thought I'd bring it up, just in case you haven't seen it. Looks like it might be right up your alley. http://dev.mysql.com/tech-resources/articles/xml-in-mysql5.1-6.0.html#xml-5. 1-in-and-out While I didn't have to process xml files, I do have to process a daily data feed of real estate data in comma-delimited files and insert that info into a MySQL 5 database. At first, I tried processing the files with CF, but it was a *really* slow process using CF, taking almost a minute to process some files. Finally, someone on this list suggested I take a look at the MySQL commands for loading data from files, such as, in your case, Load_File(), which opens an entire XML document, makes it available as a string, and inserts this string into a table column. In my case, using MySQL's load data infile, the time was reduced to less than a second. Check out the page above for details on usage and syntax. hth, Rick -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Wednesday, February 16, 2011 12:22 PM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? Hey everybody, we're in a bind here and I pitched the idea to the client of bringing in a ringer. Is there someone out there who is interested in building a direct-to-SQL Server or direct-to-mySQL-based solution? Solution would have to, on a scheduled basis, grab a file in a location on the server on its own without CF and process it (we have stuff that retrieves/places the file). Current file size is about 50 mb but must be expected to go to around 100. At 42mb using just CF I am at about 70 seconds per record loop. We need a *significant* improvement in performance. Work would be on a dedicated Windows server. Present box is a fairly powerful Crystaltech Win2003 Server w/4GB RAM and a Xeon 2.8 processor w/4 cores. We'd consider a Win2008, 64-bit installation to improve speed. Looking at a 24-core server at Viviotech w/16gb as a hi-horsepower alternative. Our feed partner is probably going to be able to use a different method of delivery that will reduce file size, but for now we need to plan for the worst and move on it. Anyone interested? You can email me at my for-reals email at matt AT mysecretbase DOT com. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342338 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Yeah I have seen that, but I'm up against a time constraint and there are a number of issues I don't have my head wrapped around, and the client needs a solution asap given that he has a daily feed encompassing thousands of records and dozens of clients ... that has now been dead about 10 days. I'm sure I could figure it out with enough time, but the client would be better served if someone who actually has done this can hit the ground running on it. BUT with this whole thing in mind, there are some things I don't get about this whole load_file() thing that maybe you or someone else can answer. I get how the file is pulled into memory, but how can a multi-level file get pulled in? Hopefully this excerpt (and its only a fragment) will explain my problem. - I don't need the header sectioon and want to skip over that. - The dataArea contains the line item records I want to pull in. - I can write CF code to pull the price and currency out of the pricing section, but how would mySQL do something like that where the xml design was made as complicated as humanly possible? (this format is StarStandard.org-based, btw) - Note the ImageAttachmentExtended records. There can be any number of those, or none. A CFloop works great but ... - the VehicleRemarketingBoat section can have any number of line items *or groups* and they vary from record to record. The groups can have more than one entry in them, or not. Here again CF gives me tools to deal with this. I just don't see how something like this can be dumped into a db at all. I'm assuming its inexperience with the task. Or is this xml a bit beyond that type of approach? headerSection headerField1blah/headerField1 headerField2blah/headerField2 headerField3yoo/headerField3 headerField4hoo/headerField4 headerField5woof/headerField5 /headerSection dataArea dataRecord recordID123/recordID year1980/year makeTrabant/make modelTurboXL/model pricing hidePricefalse/hidePrice hidePrice8/hidePrice Price ChargeAmount currencyID='USD'99500/ChargeAmount /Price /pricing Location LocationAddress CityNameAnnapolis/CityName CountryIDUS/CountryID StateOrProvinceCountrySubDivisionIDMD/StateOrProvinceCountrySubDivisionID /LocationAddress /Location ImageAttachmentExtended ID46795014/ID URIhttp://img.arf.com/images/1/70/33/3197033_0_08022024_1.jpg/URI ImageWidthMeasure unitCode='pixel'412/ImageWidthMeasure ImageHeightMeasure unitCode='pixel'471/ImageHeightMeasure ImageLastModifiedDateTime2011-02-08T11:24:00-08:00/ImageLastModifiedDateTime UsagePreference PriorityRankingNumeric1/PriorityRankingNumeric /UsagePreference /ImageAttachmentExtended ImageAttachmentExtended ID123456/ID URIhttp://img.arf.com/images/1/70/33/3197033_0_08022024_2.jpg/URI ImageWidthMeasure unitCode='pixel'412/ImageWidthMeasure ImageHeightMeasure unitCode='pixel'471/ImageHeightMeasure ImageLastModifiedDateTime2011-02-08T11:24:00-08:00/ImageLastModifiedDateTime UsagePreference PriorityRankingNumeric2/PriorityRankingNumeric /UsagePreference /ImageAttachmentExtended ImageAttachmentExtended ID987654/ID URIhttp://img.arf.com/images/1/70/33/3197033_0_08022024_3.jpg/URI ImageWidthMeasure unitCode='pixel'412/ImageWidthMeasure ImageHeightMeasure unitCode='pixel'471/ImageHeightMeasure ImageLastModifiedDateTime2011-02-08T11:24:00-08:00/ImageLastModifiedDateTime UsagePreference PriorityRankingNumeric1/PriorityRankingNumeric /UsagePreference /ImageAttachmentExtended VehicleRemarketingBoat MakeStringStarratt and Jenks/MakeString ModelYear1979/ModelYear SaleClassCodeUsed/SaleClassCode ModelMorgan 45/Model BoatLengthGroup BoatLengthCodeNominal Length/BoatLengthCode BoatLengthMeasure unitCode='feet'45/BoatLengthMeasure /BoatLengthGroup BoatLengthGroup BoatLengthCodeLength At Water Line/BoatLengthCode BoatLengthMeasure unitCode='feet'31.42/BoatLengthMeasure /BoatLengthGroup BoatLengthGroup BoatLengthCodeLength Overall/BoatLengthCode BoatLengthMeasure unitCode='feet'45/BoatLengthMeasure /BoatLengthGroup BeamMeasure unitCode='feet'11/BeamMeasure DraftMeasureGroup DraftMeasure unitCode='feet'6.08/DraftMeasure BoatDraftCodeMax Draft/BoatDraftCode /DraftMeasureGroup
Re: Big XML files processing Really s-l-o-w. Solution?
Matt, as you want a non CF solution, you might be best posting elsewhere to find a database guru who could do it all directly on the DB. If no-one here pipes up then try www.odesk.com On Wed, Feb 16, 2011 at 6:09 PM, Rick Faircloth r...@whitestonemedia.comwrote: Hi, Matt... I know you might have looked at this link at dev.mysql.com for a solution to processing your data feed, but I thought I'd bring it up, just in case you haven't seen it. Looks like it might be right up your alley. http://dev.mysql.com/tech-resources/articles/xml-in-mysql5.1-6.0.html#xml-5. 1-in-and-out While I didn't have to process xml files, I do have to process a daily data feed of real estate data in comma-delimited files and insert that info into a MySQL 5 database. At first, I tried processing the files with CF, but it was a *really* slow process using CF, taking almost a minute to process some files. Finally, someone on this list suggested I take a look at the MySQL commands for loading data from files, such as, in your case, Load_File(), which opens an entire XML document, makes it available as a string, and inserts this string into a table column. In my case, using MySQL's load data infile, the time was reduced to less than a second. Check out the page above for details on usage and syntax. hth, Rick -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Wednesday, February 16, 2011 12:22 PM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? Hey everybody, we're in a bind here and I pitched the idea to the client of bringing in a ringer. Is there someone out there who is interested in building a direct-to-SQL Server or direct-to-mySQL-based solution? Solution would have to, on a scheduled basis, grab a file in a location on the server on its own without CF and process it (we have stuff that retrieves/places the file). Current file size is about 50 mb but must be expected to go to around 100. At 42mb using just CF I am at about 70 seconds per record loop. We need a *significant* improvement in performance. Work would be on a dedicated Windows server. Present box is a fairly powerful Crystaltech Win2003 Server w/4GB RAM and a Xeon 2.8 processor w/4 cores. We'd consider a Win2008, 64-bit installation to improve speed. Looking at a 24-core server at Viviotech w/16gb as a hi-horsepower alternative. Our feed partner is probably going to be able to use a different method of delivery that will reduce file size, but for now we need to plan for the worst and move on it. Anyone interested? You can email me at my for-reals email at matt AT mysecretbase DOT com. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342343 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Mon, Feb 14, 2011 at 11:44 PM, Matt Robertson wrote: I am tasked with doing an import on a file whose size can essentially be unlimited. We've been able to handle in the 10-15mb range but it recently ballooned to 100 mb, and its going to get larger. Processing time seems to be about 66 hours for a 45 mb file and thats a disaster. Would it help if you split the file into individual records before processing them? http://jochem.vandieten.net/2011/02/17/splitting-large-xml-files-with-coldfusion-and-stax/ Jochem -- Jochem van Dieten http://jochem.vandieten.net/ ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342353 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Wed, Feb 16, 2011 at 3:49 PM, Jochem van Dieten wrote: Would it help if you split the file into individual records before processing them? I think it would help immensely I think. I'd prefer to fight my way thru this in CF if at all possible and things are just complex enough that I may need to go the route of stax. Outside my comfort zone in terms of installing it on the server but I can likely figure it out. This morning I investigated the possibility of dropping into java and reading one line at a time, parsing out the text in that line... Its what I do in this file's CSV-format predecessor. But there are no CR's or LF's in the file so that is out. Thanks very much for that post and your thoughts. Incidentally if anyone is looking for an xml reader/editor that can almost instantly read these kinds of moster files, check out the free firstobject xml editor. http://www.firstobject.com/ -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342360 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
Hi, Matt...
I've been tinkering with parsing the xml file and, though
I haven't developed a full solution, I've made some progress
that I thought I'd share.
I created a MySQL 5 database called, xmlTest.
I created a MySQL 5 database table called, xmlBlob.
Fields:
- rowID (int, not null, auto increment)
- xmlContent (blob)
- comment (varchar, 200)
I then ran this code:
cfset filePath = #expandPath('data.xml')#
cfoutputfilePath = #filePath#/cfoutput (checking accuracy)
cfquery name=processXML datasource=xmlTest
select xmlContent
fromxmlBlob
limit 1
into@xml
/cfquery
Data extraction method 1:
cfquery name=extractData01 datasource=xmlTest
select extractValue(@xml, '//dataRecord[1]/recordID[1]')
recordID
/cfquery
pcfdump toString(extractData01.recordID) = cfdump
var=#toString(extractData01.recordID)#/p
Data extraction method 2:
cfquery name=extractData02 datasource=xmlTest
select extractValue(b.xmlContent, '//dataRecord/recordID')
recordID,
extractValue(b.xmlContent,
'//dataRecord/year') year,
extractValue(b.xmlContent,
'//dataRecord/make') make,
extractValue(b.xmlContent,
'//dataRecord/model') model
fromxmlBlob b
/cfquery
cfoutputextractData02.recordCount =
#extractData02.recordCount#/cfoutputbr
br
cfoutput query=extractData02
#toString(extractData02.recordID)#br
#toString(extractData02.year)#br
#toString(extractData02.make)#br
#toString(extractData02.model)#br
br
/cfoutput
---
When the code above is run in a browser, it outputs:
cfdump toString(extractData01.recordID) = 123
extractData02.recordCount = 1
123 124
1980 1981
Trabant Ford
TurboXL TurboDL
---
I was just to the point of figuring out how to get
the two records (I modified your data.xml file example so there
would be two records) to show separately, instead of
the data being in a list for the fields, when I saw your
post and thought I'd go ahead and respond so you could see
if this might be an approach that would work for you.
In the data extraction method 1 above, the idea is to insert
the xml file (which might be too large for this?)
into a blob field in a MySQL table, then read
the blob field and put the xml data file into a variable, '@xml',
then run a cfquery against the variable.
Data extraction method 2 skips reading the xml into a variable
and queries the blob field directly. This method uses the
MySQL 'xpath_expression' syntax to access data in multi-level
xml data files. With xpath, you can go as deeply into the levels
as needed.
Like I said, this is just the result of tinkering around with
what I've been finding and attempting to get some usable output.
Perhaps this will be of some help.
Rick
-Original Message-
From: Matt Robertson [mailto:websitema...@gmail.com]
Sent: Wednesday, February 16, 2011 10:18 PM
To: cf-talk
Subject: Re: Big XML files processing Really s-l-o-w. Solution?
On Wed, Feb 16, 2011 at 3:49 PM, Jochem van Dieten wrote:
Would it help if you split the file into individual records before
processing them?
I think it would help immensely I think. I'd prefer to fight my way
thru this in CF if at all possible and things are just complex enough
that I may need to go the route of stax. Outside my comfort zone in
terms of installing it on the server but I can likely figure it out.
This morning I investigated the possibility of dropping into java and
reading one line at a time, parsing out the text in that line... Its
what I do in this file's CSV-format predecessor. But there are no
CR's or LF's in the file so that is out.
Thanks very much for that post and your thoughts.
Incidentally if anyone is looking for an xml reader/editor that can
almost instantly read these kinds of moster files, check out the free
firstobject xml editor.
http://www.firstobject.com/
--
--m@Robertson--
Janitor, The Robertson Team
mysecretbase.com
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342361
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Big XML files processing Really s-l-o-w. Solution?
My question is how do I reduce processing time... is there anything I
could be doing better?
I am tasked with doing an import on a file whose size can essentially
be unlimited. We've been able to handle in the 10-15mb range but it
recently ballooned to 100 mb, and its going to get larger. Processing
time seems to be about 66 hours for a 45 mb file and thats a disaster.
For a 14mb file its about 90 minutes.
Whats happening is this: CF is looping over a very large number of
records that are in themselves fairly complex. The more records there
are, the longer the loop over them takes. More records + more time
per record = a seemingly geometric increase in processing time.
The file is read in like so.
!---
read the imported file
---
cflock
name=mates_import
type=EXCLUSIVE
timeout=10
cffile
action=READ
file=#variables.mates.srcFile#
variable=x
/cflock
cfset x =
replaceNoCase(x,StateOrProvinceCountrySub-DivisionID,StateProvince,ALL)
!---
turn the file into a coldfusion xml object
---
cfset x=ltrim(trim(x))
cfset x=XMLParse(x)
The above takes only a few seconds. No problem there.
Next I have to read in some header info
cfscript:
header.logicalID=x.processvehicleremarketing.applicationarea.sender.logicalID.xmltext;
header.taskID=x.processvehicleremarketing.applicationarea.sender.taskID.xmltext;
header.BODID=x.processvehicleremarketing.applicationarea.BODID.xmltext;
header.created=x.processvehicleremarketing.applicationarea.CreationDateTime.xmltext;
// ...
// and here comes the node with all of the line items in it I'll have
to loop over. This is where all of the speed issues have been traced
to:
variables.mates.boatArrayLen=arrayLen(x.processvehicleremarketing.ProcessVehicleRemarketingDataArea.VehicleRemarketing);
/cfscript
knowing the array length I can use CFLOOP to look over it and pull
data in where it is then stored in a db.
cfloop
from=1
to=#variables.mates.boatArrayLen#
index=i
cfscript
listings_mates.inHouseListingNumber=M-
x.processVehicleRemarketing.processVehicleRemarketingDataArea.vehicleRemarketing[i].vehicleRemarketingHeader.documentIdentificationGroup.documentIdentification.documentID.xmltext;
listings_mates.price=x.processVehicleRemarketing.processVehicleRemarketingDataArea.vehicleRemarketing[i].vehicleRemarketingBoatLineItem.pricingABIE.price.chargeAmount.xmltext;
listings_mates.currency=x.processVehicleRemarketing.processVehicleRemarketingDataArea.vehicleRemarketing[i].vehicleRemarketingBoatLineItem.pricingABIE.price.chargeAmount.xmlAttributes.currencyID;
// there can be more than one of these items so run a loop inside
of the loop
variables.mates.engineArrayLen=arrayLen(x.processVehicleRemarketing.processVehicleRemarketingDataArea.vehicleRemarketing[i].vehicleRemarketingBoatLineItem.VehicleRemarketingEngineLineItem);
ii=0;
do {
ii=ii+1;
listings_mates.engineDesc=x.processVehicleRemarketing.processVehicleRemarketingDataArea.vehicleRemarketing[i].vehicleRemarketingBoatLineItem.VehicleRemarketingEngineLineItem[ii].VehicleRemarketingEngine.modelDescription.xmltext;
listings_mates.engHrs=x.processVehicleRemarketing.processVehicleRemarketingDataArea.vehicleRemarketing[i].vehicleRemarketingBoatLineItem.VehicleRemarketingEngineLineItem[ii].VehicleRemarketingEngine.totalEngineHoursNumeric.xmltext;
} while (ii LTE variables.mates.engineArrayLen);
...
/cfscript
...
/cfloop
And so on. A hundred or so fields and a dozen or so loops inside the
main loop, along with a loop or two inside of those. So the very long
variable names get even longer.
As you can see I am pouring the data into a struct, and when done, I
insert it as a db record.
Anyone see a mistake in my methods? Would things speed up if, before
I read it into an xml object, I ran a replace() or three to shorten up
some of those names?
--
--m@Robertson--
Janitor, The Robertson Team
mysecretbase.com
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342213
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Mon, Feb 14, 2011 at 5:44 PM, Matt Robertson websitema...@gmail.comwrote: My question is how do I reduce processing time... is there anything I could be doing better? I am tasked with doing an import on a file whose size can essentially be unlimited. We've been able to handle in the 10-15mb range but it recently ballooned to 100 mb, and its going to get larger. Processing time seems to be about 66 hours for a 45 mb file and thats a disaster. For a 14mb file its about 90 minutes. Whats happening is this: CF is looping over a very large number of records that are in themselves fairly complex. The more records there are, the longer the loop over them takes. More records + more time per record = a seemingly geometric increase in processing time. Are you using MS SQL 2005? Because throwing the XML file directly at SQL server works a 1000% better. I've basically given up on processing large XML files directly with coldfusion. IF you can save these xml files on the SQL server you can make use of BulkInsert but that needs to read the file from that server. OR just setup a stored procedure, send your xml over and let it do it's thing. http://msdn.microsoft.com/en-us/library/ms345117%28v=SQL.90%29.aspx ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342215 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Either that, or start looking at something like StaX - http://stax.codehaus.org/Home Pulling that large a file in memory is going to suck a lot, no matter what you do. Mark On Tue, Feb 15, 2011 at 10:24 AM, Casey Dougall ca...@uberwebsitesolutions.com wrote: On Mon, Feb 14, 2011 at 5:44 PM, Matt Robertson websitema...@gmail.comwrote: My question is how do I reduce processing time... is there anything I could be doing better? I am tasked with doing an import on a file whose size can essentially be unlimited. We've been able to handle in the 10-15mb range but it recently ballooned to 100 mb, and its going to get larger. Processing time seems to be about 66 hours for a 45 mb file and thats a disaster. For a 14mb file its about 90 minutes. Whats happening is this: CF is looping over a very large number of records that are in themselves fairly complex. The more records there are, the longer the loop over them takes. More records + more time per record = a seemingly geometric increase in processing time. Are you using MS SQL 2005? Because throwing the XML file directly at SQL server works a 1000% better. I've basically given up on processing large XML files directly with coldfusion. IF you can save these xml files on the SQL server you can make use of BulkInsert but that needs to read the file from that server. OR just setup a stored procedure, send your xml over and let it do it's thing. http://msdn.microsoft.com/en-us/library/ms345117%28v=SQL.90%29.aspx ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342216 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Mon, Feb 14, 2011 at 6:27 PM, Mark Mandel mark.man...@gmail.com wrote: Either that, or start looking at something like StaX - http://stax.codehaus.org/Home Pulling that large a file in memory is going to suck a lot, no matter what you do. Mark yeah, I don't know, still seems better equipped for SQL server if you can do it there. cfstoredproc procedure=spBIG_XML_Insert datasource=#request.DataSource# returncode=Yes cfprocparam type=In cfsqltype=CF_SQL_VARCHAR variable=mydoc value=#r.SXMLRESPONSE# /cfstoredproc And then the trimmed down Stored Procedure ALTER PROCEDURE [dbo].[spBIG_XML_Insert] @mydoc xml, AS declare @hdoc int -- Create an internal representation of the XML document. EXEC sp_xml_preparedocument @hdoc OUTPUT, @mydoc, 'soap:Envelope xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; xmlns:xsd=http://www.w3.org/2001/XMLSchema; xmlns:soap=http://schemas.xmlsoap.org/soap/envelope/; xmlns:c=urn:Scores/Services/v2/ScoresUpdate /' -- Now do the insert INSERT INTO tblScoreUpdates SELECT MemberID = x.MemberID, ReportDate= x.ReportDate, Score = x.Score, DateAdded = GetDate() FROM OPENXML (@hdoc, 'soap:Envelope/soap:Body/c:LatestScoresResponse/c:parameters/c:Scores',2) WITH ( MemberID varchar(10) '@MemberID', ReportDate varchar(25) '@ReportDate', Scoreint '@Score') as x WHERE x.MemberID NOT IN (SELECT t.MemberID FROM tblScoreUpdates t With (NoLock) WHERE t.MemberID= x.MemberID) ; --Do another query here if you want, --remove the document from memory EXEC sp_xml_removedocument @hDoc And if you needed something from a level up... just dot notation back to it. LogID = int '../@LogID which would grab the logID from Parameters... ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342218 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Client is running mySQL, But a SQL Server could probably be arranged if we have to. Kind of a painful investment for them, though. I was kind of hoping you guys would find a bonehead mistake that would magically clear this up. I'm out of CF-based ideas. Speaking of which, I tried shortening those var names with some replace() statements and it had zero effect. Didn't expect it would but wanted to throw it up against the wall. Damn shame I can't drop to java and do a line-by-line read like I do with CSV files :-( -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342220 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Mon, Feb 14, 2011 at 3:27 PM, Mark Mandel wrote: Either that, or start looking at something like StaX - http://stax.codehaus.org/Home thx for the tip on Stax but it frankly looks a bit out of my league for this project. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342221 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
