Re: [osg-users] osgDB XmlParser and localization
Thanks a bunch Robert! I will check it first thing tomorrow morning On Tue, Sep 4, 2018 at 4:40 PM Robert Osfield wrote: > On Fri, 31 Aug 2018 at 08:06, Trajce Nikolov NICK > wrote: > > can you fix this too when you get back to OSG dev please? > > This morning I have checked in UTF8 handling in XmlNode/Input to > master and the 3.6 branch. > > > https://github.com/openscenegraph/OpenSceneGraph/commit/afe5644b9fcf39a22a2ba0c8aefb26a340a5a457 > > Robert. > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > -- trajce nikolov nick ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] osgDB XmlParser and localization
On Fri, 31 Aug 2018 at 08:06, Trajce Nikolov NICK wrote: > can you fix this too when you get back to OSG dev please? This morning I have checked in UTF8 handling in XmlNode/Input to master and the 3.6 branch. https://github.com/openscenegraph/OpenSceneGraph/commit/afe5644b9fcf39a22a2ba0c8aefb26a340a5a457 Robert. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] osgDB XmlParser and localization
Hi Robert, can you fix this too when you get back to OSG dev please? Thank you a bunch Nick On Mon, Jul 9, 2018 at 1:58 PM Trajce Nikolov NICK < trajce.nikolov.n...@gmail.com> wrote: > Hi again Robert, > > fast hint: The nodes to be parsed are root->osm->node->tag and their > properties contains these non asci codes. But probably fastest is to write > recursive parser > > Thanks again > > On Mon, Jul 9, 2018 at 1:54 PM Trajce Nikolov NICK < > trajce.nikolov.n...@gmail.com> wrote: > >> Thanks so much Robert !!! Attached is the file >> >> >> >> On Mon, Jul 9, 2018 at 1:50 PM Robert Osfield >> wrote: >> >>> Hi Nick, >>> >>> On Mon, 9 Jul 2018 at 12:26, Trajce Nikolov NICK >>> wrote: >>> > > are changes for changes sake rather than likely to make any >>> > functional difference >>> > >>> > I knew you will have comments ;-). It works just fine with these >>> changes but yes, you are right - quick and somewhat dirty. So can you look >>> at it or you want to do a ping-pong code review with my changes by your >>> advice? :-) >>> >>> I might be quicker just to provide a link to or attach an xml file >>> that causes problem with the present parser, and then I can look at >>> what happens when the present code encounters it, and how your changes >>> affect things. >>> >>> As a general note, PRs/commits should be focused on addressing one >>> issue at a time, so bundling a fix for a specific bug along with other >>> cosmetic or other changes just confuses what is functional and what is >>> not. It's not uncommon to have to go back to various revisions in >>> code to see where regressions may have occurred so having commits that >>> just wrap up a small set of associated changes is really helpful in >>> identifying what has been changed and why. >>> >>> Robert. >>> ___ >>> osg-users mailing list >>> osg-users@lists.openscenegraph.org >>> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org >>> >> >> >> -- >> trajce nikolov nick >> > > > -- > trajce nikolov nick > -- trajce nikolov nick ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] osgDB XmlParser and localization
Hi again Robert, fast hint: The nodes to be parsed are root->osm->node->tag and their properties contains these non asci codes. But probably fastest is to write recursive parser Thanks again On Mon, Jul 9, 2018 at 1:54 PM Trajce Nikolov NICK < trajce.nikolov.n...@gmail.com> wrote: > Thanks so much Robert !!! Attached is the file > > > > On Mon, Jul 9, 2018 at 1:50 PM Robert Osfield > wrote: > >> Hi Nick, >> >> On Mon, 9 Jul 2018 at 12:26, Trajce Nikolov NICK >> wrote: >> > > are changes for changes sake rather than likely to make any >> > functional difference >> > >> > I knew you will have comments ;-). It works just fine with these >> changes but yes, you are right - quick and somewhat dirty. So can you look >> at it or you want to do a ping-pong code review with my changes by your >> advice? :-) >> >> I might be quicker just to provide a link to or attach an xml file >> that causes problem with the present parser, and then I can look at >> what happens when the present code encounters it, and how your changes >> affect things. >> >> As a general note, PRs/commits should be focused on addressing one >> issue at a time, so bundling a fix for a specific bug along with other >> cosmetic or other changes just confuses what is functional and what is >> not. It's not uncommon to have to go back to various revisions in >> code to see where regressions may have occurred so having commits that >> just wrap up a small set of associated changes is really helpful in >> identifying what has been changed and why. >> >> Robert. >> ___ >> osg-users mailing list >> osg-users@lists.openscenegraph.org >> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org >> > > > -- > trajce nikolov nick > -- trajce nikolov nick ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] osgDB XmlParser and localization
Hi Nick, On Mon, 9 Jul 2018 at 12:26, Trajce Nikolov NICK wrote: > > are changes for changes sake rather than likely to make any > functional difference > > I knew you will have comments ;-). It works just fine with these changes but > yes, you are right - quick and somewhat dirty. So can you look at it or you > want to do a ping-pong code review with my changes by your advice? :-) I might be quicker just to provide a link to or attach an xml file that causes problem with the present parser, and then I can look at what happens when the present code encounters it, and how your changes affect things. As a general note, PRs/commits should be focused on addressing one issue at a time, so bundling a fix for a specific bug along with other cosmetic or other changes just confuses what is functional and what is not. It's not uncommon to have to go back to various revisions in code to see where regressions may have occurred so having commits that just wrap up a small set of associated changes is really helpful in identifying what has been changed and why. Robert. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] osgDB XmlParser and localization
hi Robert, > are changes for changes sake rather than likely to make any functional difference I knew you will have comments ;-). It works just fine with these changes but yes, you are right - quick and somewhat dirty. So can you look at it or you want to do a ping-pong code review with my changes by your advice? :-) Please let me know Nick On Mon, Jul 9, 2018 at 1:09 PM Robert Osfield wrote: > Hi Nick, > > I had a quick look at your changes and it seems to be like most of > them are changes for changes sake rather than likely to make any > functional difference. For non ascii char support I think what you'd > actually want to do is change the XmllNode::Input::string _buffer > member var to a std::vector _buffer, and have the > Input::readAllDataIntoBuffer() method be adapted to read the file > accepting only chars greater than 0 rather than limiting to the ascii > 0 to 255 range. > > Robert. > On Mon, 9 Jul 2018 at 11:41, Trajce Nikolov NICK > wrote: > > > > Hi Robert, > > > > I am working with some localized XML files (with chars out of 0-255 > range) and at present the XML Node parsing is not suitable to manage it. > And I think the support for is important - at my case it is OpenStreetMap > with street names containing these chars. > > > > Attached is the modified source (based on the master) that fixes ity, if > you have time and will to review. I can do PR if you want > > > > Thanks and cheers! > > > > Nick > > > > p.s. welcome back > > > > -- > > trajce nikolov nick > > ___ > > osg-users mailing list > > osg-users@lists.openscenegraph.org > > > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > -- trajce nikolov nick ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] osgDB XmlParser and localization
Hi Nick, I had a quick look at your changes and it seems to be like most of them are changes for changes sake rather than likely to make any functional difference. For non ascii char support I think what you'd actually want to do is change the XmllNode::Input::string _buffer member var to a std::vector _buffer, and have the Input::readAllDataIntoBuffer() method be adapted to read the file accepting only chars greater than 0 rather than limiting to the ascii 0 to 255 range. Robert. On Mon, 9 Jul 2018 at 11:41, Trajce Nikolov NICK wrote: > > Hi Robert, > > I am working with some localized XML files (with chars out of 0-255 range) > and at present the XML Node parsing is not suitable to manage it. And I think > the support for is important - at my case it is OpenStreetMap with street > names containing these chars. > > Attached is the modified source (based on the master) that fixes ity, if you > have time and will to review. I can do PR if you want > > Thanks and cheers! > > Nick > > p.s. welcome back > > -- > trajce nikolov nick > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
[osg-users] osgDB XmlParser and localization
Hi Robert,
I am working with some localized XML files (with chars out of 0-255 range)
and at present the XML Node parsing is not suitable to manage it. And I
think the support for is important - at my case it is OpenStreetMap with
street names containing these chars.
Attached is the modified source (based on the master) that fixes ity, if
you have time and will to review. I can do PR if you want
Thanks and cheers!
Nick
p.s. welcome back
--
trajce nikolov nick
/* -*-c++-*- OpenSceneGraph - Copyright (C) 1998-2009 Robert Osfield
*
* This library is open source and may be redistributed and/or modified under
* the terms of the OpenSceneGraph Public License (OSGPL) version 0.0 or
* (at your option) any later version. The full license is in LICENSE file
* included with this distribution, and on the openscenegraph.org website.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* OpenSceneGraph Public License for more details.
*/
#include
#include
#include
using namespace osgDB;
XmlNode* osgDB::readXmlFile(const std::string& filename,const Options* options)
{
std::string foundFile = osgDB::findDataFile(filename, options);
if (!foundFile.empty())
{
XmlNode::Input input;
input.open(foundFile);
input.readAllDataIntoBuffer();
if (!input)
{
OSG_NOTICE<<"Could not open XML file: "< root = new XmlNode;
root->read(input);
return root.release();
}
else
{
OSG_NOTICE<<"Could not find XML file: "< root = new XmlNode;
root->read(input);
return root.release();
}
XmlNode::ControlMap::ControlMap()
{
setUpControlMappings();
}
void XmlNode::ControlMap::addControlToCharacter(const std::string& control, int
c)
{
_controlToCharacterMap[control] = c;
_characterToControlMap[c] = control;
}
void XmlNode::ControlMap::setUpControlMappings()
{
addControlToCharacter("",'&');
addControlToCharacter("",'<');
addControlToCharacter("",'>');
addControlToCharacter("",'"');
addControlToCharacter("",'\'');
addControlToCharacter("",'\n');
}
XmlNode::Input::Input():
_currentPos(0)
{
}
XmlNode::Input::Input(const Input&):
ControlMap(),
_currentPos(0)
{
}
XmlNode::Input::~Input()
{
}
void XmlNode::Input::open(const std::string& filename)
{
_fin.open(filename.c_str());
}
void XmlNode::Input::attach(std::istream& fin)
{
std::ios = _fin;
fios.rdbuf(fin.rdbuf());
}
void XmlNode::Input::readAllDataIntoBuffer()
{
while(_fin)
{
int c = _fin.get();
//if (c>=0 && c<=255)
{
_buffer.push_back(c);
}
}
}
void XmlNode::Input::skipWhiteSpace()
{
while(_currentPos<_buffer.size() && (_buffer[_currentPos]==' ' ||
_buffer[_currentPos]=='\t' || _buffer[_currentPos]=='\n' ||
_buffer[_currentPos]=='\r'))
{
//OSG_NOTICE<<"_currentPos="<<_currentPos<<"_buffer.size()="<<_buffer.size()<<"
v="");
commentNode->contents = input.substr(0, end);
if (end!=std::string::npos)
{
OSG_INFO<<"Valid Comment record
["");
commentNode->contents = input.substr(0, end);
if (end!=std::string::npos)
{
OSG_INFO<<"Valid information record
["");
commentNode->contents = input.substr(0, end);
if (end!=std::string::npos)
{
OSG_INFO<<"Valid information record
["");
commentNode->contents = input.substr(0, end);
if (end!=std::string::npos)
{
OSG_INFO<<"Valid information record
["name.push_back(c);
