On 22 March 2013 23:21, Andrea Di Menna <ninn...@gmail.com> wrote: > > Hi Jona, > > thanks for merging the pull request! > > Anyway, couldn't we use percent encoding for Unicode code points which are > not allowed in N-Triples? (namely those outside the [#x20,#7E] range? > In this case we should get UTF-8 bytes and percent encode them. > > For example, as far as I can see > > Marl$00C3$00ADn$002C_$00C3$0081vila > > is > > <http://dbpedia.org/resource/Marl%C3%83%C2%ADn,_%C3%83%C2%81vila> > > where \00C3 is 0xC3 0x83 > \00AD is 0xC2 0xAD > \0081 is 0xC2 0x81 > > WDYT?
I prefer the "garbage in, garbage out" style. The freebase keys are broken. We could try to fix them, but we would have to use several different heuristics: with percent encoding, we could "fix" the keys that are UTF-8 encoded, but not the ones that are Windows-encoded. To fix the keys containing whitespace, we would first have to UTF-8 the Unicode code point, then percent encode the UTF-8... it's a mess. And anyway, we try to move towards IRIs, not URIs, and IRIs wouldn't contain percent-encodings for these characters. How many keys are affected anyway? I think we generate several million freebase links, so even if 100,000 freebase keys are broken, it's not a big problem. JC > > Cheers > Andrea > > 2013/3/22 Christopher Sahnwaldt <notificati...@github.com> >> >> Ok, I got it. It has nothing to do with your platform. These are actually >> wrong URIs. There's not much we can do about it. I don't know where Freebase >> got them from, but I assume they may actually be wrong in Wikipedia. >> >> Examples: >> >> Marl$00C3$00ADn$002C_$00C3$0081vila >> AD 2C and C3 81 are UTF-8 encodings, but Freebase says [1] that the >> numbers should be plain Unicode code points, not UTF-8 bytes. 81 is an >> invalid code point, so we generate an invalid URI. >> >> Bene$009A_decrees >> 9A is the Windows-1252 encoding for š, but 9A invalid in Unicode. >> >> Switzerland$2003 >> 2003, 2029 etc. are valid Unicode code points, but for whitespace >> characters that are invalid in URIs >> >> In a nutshell: all these characters are invalid in URIs, and it's not our >> fault. I'll pull your changes in a moment. >> >> [1] http://wiki.freebase.com/wiki/MQL_key_escaping >> >> — >> Reply to this email directly or view it on GitHub. > > ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar _______________________________________________ Dbpedia-discussion mailing list Dbpedia-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
