Re: [External Email] Re: [datameet] Creating variable names in Devanagari

['Shiv Hastawala' via datameet]

Hi Sharad

I'm sorry, I haven't ever worked with shapefiles or geopkg files so I'm
afraid I don't have expertise in this area. I just had access to these data
files, so I passed them on.

On Wed, 6 Dec 2023 at 8:19 am, Sharad Lele  wrote:

> Hi Nikhil, Ujaval, and others who responded:
> Shiv, my colleague who handles our webgis, tried importing the geopkg
> version (which was storing and displaying Devanagari variable names in
> QGIS) into postGIS. He says "I used the normal pguploader, shape2posrgis,
> and ogr2ogr method". But reports that postGIS does not accept those
> variables.
>
> Any workarounds?
>
> Sharad
>
> On Tue, 5 Dec 2023, 17:28 Sharad Lele (शरच्चंद्र लेले), <
> sharad.l...@gmail.com> wrote:
>
>> Ok, thanks, Nikhil. Let me see what works at our backend.
>>
>> Sharad
>>
>> On 05-Dec-23 16:51, Nikhil VJ wrote:
>>
>> Hi Sharad,
>>
>> If you have to import the data into a web-based backend, then your
>> programmers will be most comfortable with: GeoJSON format.
>> Because : It's a simple text/json format which can even be opened as raw
>> text, and so there are many different ways to load it in a program and use
>> data from it.
>> And this will also support Devnagri columns (which become property keys
>> here) same as geopackage.
>>
>> Another thing to look out for: Ensure that the database being used by the
>> backend has the same UTF-8 encoding set.
>>
>> --
>> Cheers,
>> Nikhil VJ
>> https://nikhilvj.co.in
>>
>>
>> On Mon, Dec 4, 2023 at 3:56 PM Sharad Lele  wrote:
>>
>>> Thanks, Ujaval. Yes saving as .geopkg works (saves the field names in
>>> full and displays properly upon reopening). This is the only solution that
>>> has worked so far.
>>>
>>> But I am not sure this is going to work for us in this particular
>>> situation where we are then using the file in postGIS to display in a
>>> webGIS. Let me ask our webGIS person.
>>>
>>> Sharad
>>>
>>> On Monday, December 4, 2023 at 3:10:18 PM UTC+5:30 Ujaval Gandhi wrote:
>>>
 Use the geopackage format. It's a much better modern format without the
 limitations of shapefiles. The default encoding would be UTF-8 and it
 should work. Tested with a new geopackage layer and it is able to create
 column names and values in Devanagri script
 https://i.imgur.com/2prELf6.png
 
 ---
 Ujaval Gandhi
 Spatial Thoughts
 www.spatialthoughts.com
 



 On Mon, Dec 4, 2023 at 2:36 PM Sharad Lele  wrote:

> The shapefile encoding is set to UTF-8.  But any edits to variable
> names only last as long as I don't save the file. Save and reopening
> results in going back to the same truncated/funny display as above [image:
> Screenshot Layer properties.jpeg]
>
> On Monday, December 4, 2023 at 2:17:41 PM UTC+5:30 Dilawar Singh wrote:
>
>> I found this
>> https://gis.stackexchange.com/questions/381308/how-to-change-qgis-3-default-encoding.
>>
>>
>> The file you are loading may have a different encoding that QGIS3.
>> You have to verify this manually and make sure that QGIS3 and file 
>> encoding
>> is the same.
>>
>> I am guessing the while QGIS3 is using utf-8, the file that you are
>> loading may be utf-16 encoded. Not sure if there are automated tools that
>> can convert UTF-16 files to UTF8 file while preserving unicode (
>> https://superuser.com/questions/1786434/convert-utf-16-le-to-utf-8-in-windows-via-command-line
>> ).
>>
>> best,
>>Dilawar
>>
>>
>>
>>  On Mon, 04 Dec 2023 14:05:01 +0530 *Sharad Lele
>> * wrote ---
>>
>> Just to clarify/muddy the situation: I am able to enter long
>> Devanagari text in the District/Jila column without any problems. It is 
>> the
>> column/field name itself that I run into problems, which may have to do
>> with the 8 characteri limit?
>>
>> [image: Screenshot QGIS devanagari field names.jpeg]
>>
>>
>> On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30 Sharad Lele wrote:
>>
>> Thanks, Dilawar. The article was very interesting/useful. But for a
>> simpleton like me, the question then is: How do I ensure that QGIS3 in
>> particular is using UTF-8 encoding every time?
>>
>> Sharad
>>
>> On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh
>> wrote:
>>
>> The answer is not going to be simple.
>>
>> Make sure that every time you save or open the file, the application
>> uses encoding utf-8. Microsoft uses another weird encoding called utf-16 
>> (
>> https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if
>> you wan

Re: [datameet] Creating variable names in Devanagari

[Sharad Lele]

Hi Nikhil, Ujaval, and others who responded:
Shiv, my colleague who handles our webgis, tried importing the geopkg
version (which was storing and displaying Devanagari variable names in
QGIS) into postGIS. He says "I used the normal pguploader, shape2posrgis,
and ogr2ogr method". But reports that postGIS does not accept those
variables.

Any workarounds?

Sharad

On Tue, 5 Dec 2023, 17:28 Sharad Lele (शरच्चंद्र लेले), <
sharad.l...@gmail.com> wrote:

> Ok, thanks, Nikhil. Let me see what works at our backend.
>
> Sharad
>
> On 05-Dec-23 16:51, Nikhil VJ wrote:
>
> Hi Sharad,
>
> If you have to import the data into a web-based backend, then your
> programmers will be most comfortable with: GeoJSON format.
> Because : It's a simple text/json format which can even be opened as raw
> text, and so there are many different ways to load it in a program and use
> data from it.
> And this will also support Devnagri columns (which become property keys
> here) same as geopackage.
>
> Another thing to look out for: Ensure that the database being used by the
> backend has the same UTF-8 encoding set.
>
> --
> Cheers,
> Nikhil VJ
> https://nikhilvj.co.in
>
>
> On Mon, Dec 4, 2023 at 3:56 PM Sharad Lele  wrote:
>
>> Thanks, Ujaval. Yes saving as .geopkg works (saves the field names in
>> full and displays properly upon reopening). This is the only solution that
>> has worked so far.
>>
>> But I am not sure this is going to work for us in this particular
>> situation where we are then using the file in postGIS to display in a
>> webGIS. Let me ask our webGIS person.
>>
>> Sharad
>>
>> On Monday, December 4, 2023 at 3:10:18 PM UTC+5:30 Ujaval Gandhi wrote:
>>
>>> Use the geopackage format. It's a much better modern format without the
>>> limitations of shapefiles. The default encoding would be UTF-8 and it
>>> should work. Tested with a new geopackage layer and it is able to create
>>> column names and values in Devanagri script
>>> https://i.imgur.com/2prELf6.png
>>> 
>>> ---
>>> Ujaval Gandhi
>>> Spatial Thoughts
>>> www.spatialthoughts.com
>>> 
>>>
>>>
>>>
>>> On Mon, Dec 4, 2023 at 2:36 PM Sharad Lele  wrote:
>>>
 The shapefile encoding is set to UTF-8.  But any edits to variable
 names only last as long as I don't save the file. Save and reopening
 results in going back to the same truncated/funny display as above [image:
 Screenshot Layer properties.jpeg]

 On Monday, December 4, 2023 at 2:17:41 PM UTC+5:30 Dilawar Singh wrote:

> I found this
> https://gis.stackexchange.com/questions/381308/how-to-change-qgis-3-default-encoding.
>
>
> The file you are loading may have a different encoding that QGIS3. You
> have to verify this manually and make sure that QGIS3 and file encoding is
> the same.
>
> I am guessing the while QGIS3 is using utf-8, the file that you are
> loading may be utf-16 encoded. Not sure if there are automated tools that
> can convert UTF-16 files to UTF8 file while preserving unicode (
> https://superuser.com/questions/1786434/convert-utf-16-le-to-utf-8-in-windows-via-command-line
> ).
>
> best,
>Dilawar
>
>
>
>  On Mon, 04 Dec 2023 14:05:01 +0530 *Sharad Lele
> * wrote ---
>
> Just to clarify/muddy the situation: I am able to enter long
> Devanagari text in the District/Jila column without any problems. It is 
> the
> column/field name itself that I run into problems, which may have to do
> with the 8 characteri limit?
>
> [image: Screenshot QGIS devanagari field names.jpeg]
>
>
> On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30 Sharad Lele wrote:
>
> Thanks, Dilawar. The article was very interesting/useful. But for a
> simpleton like me, the question then is: How do I ensure that QGIS3 in
> particular is using UTF-8 encoding every time?
>
> Sharad
>
> On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh wrote:
>
> The answer is not going to be simple.
>
> Make sure that every time you save or open the file, the application
> uses encoding utf-8. Microsoft uses another weird encoding called utf-16 (
> https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if
> you want to copy-paste anything from the web or web-based tools.
>
> Long answer:
> https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/
>
> best,
>Dilawar
>
>
>
>  On Mon, 04 Dec 2023 13:24:28 +0530 *Sharad Lele
> * wrote ---
>
>

Re: [datameet] Creating variable names in Devanagari

[शरच्चंद्र लेले]

Ok, thanks, Nikhil. Let me see what works at our backend.

Sharad


On 05-Dec-23 16:51, Nikhil VJ wrote:

Hi Sharad,

If you have to import the data into a web-based backend, then your 
programmers will be most comfortable with: GeoJSON format.
Because : It's a simple text/json format which can even be opened as 
raw text, and so there are many different ways to load it in a program 
and use data from it.
And this will also support Devnagri columns (which become property 
keys here) same as geopackage.


Another thing to look out for: Ensure that the database being used by 
the backend has the same UTF-8 encoding set.


--
Cheers,
Nikhil VJ
https://nikhilvj.co.in


On Mon, Dec 4, 2023 at 3:56 PM Sharad Lele > wrote:


Thanks, Ujaval. Yes saving as .geopkg works (saves the field names
in full and displays properly upon reopening). This is the only
solution that has worked so far.

But I am not sure this is going to work for us in this particular
situation where we are then using the file in postGIS to display
in a webGIS. Let me ask our webGIS person.

Sharad

On Monday, December 4, 2023 at 3:10:18 PM UTC+5:30 Ujaval Gandhi
wrote:

Use the geopackage format. It's a much better modern format
without the limitations of shapefiles. The default encoding
would be UTF-8 and it should work. Tested with a new
geopackage layer and it is able to create column names and
values in Devanagri script https://i.imgur.com/2prELf6.png


---
Ujaval Gandhi
Spatial Thoughts
www.spatialthoughts.com





On Mon, Dec 4, 2023 at 2:36 PM Sharad Lele
 wrote:

The shapefile encoding is set to UTF-8.  But any edits to
variable names only last as long as I don't save the file.
Save and reopening results in going back to the same
truncated/funny display as above Screenshot Layer
properties.jpeg

On Monday, December 4, 2023 at 2:17:41 PM UTC+5:30 Dilawar
Singh wrote:

I found this

https://gis.stackexchange.com/questions/381308/how-to-change-qgis-3-default-encoding.


The file you are loading may have a different encoding
that QGIS3. You have to verify this manually and make
sure that QGIS3 and file encoding is the same.

I am guessing the while QGIS3 is using utf-8, the file
that you are loading may be utf-16 encoded. Not sure
if there are automated tools that can convert UTF-16
files to UTF8 file while preserving unicode

(https://superuser.com/questions/1786434/convert-utf-16-le-to-utf-8-in-windows-via-command-line).

best,
   Dilawar



 On Mon, 04 Dec 2023 14:05:01 +0530 *Sharad Lele
* wrote ---

Just to clarify/muddy the situation: I am able to
enter long Devanagari text in the District/Jila
column without any problems. It is the
column/field name itself that I run into problems,
which may have to do with the 8 characteri limit?

Screenshot QGIS devanagari field names.jpeg


On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30
Sharad Lele wrote:

Thanks, Dilawar. The article was very
interesting/useful. But for a simpleton like
me, the question then is: How do I ensure that
QGIS3 in particular is using UTF-8 encoding
every time?

Sharad

On Monday, December 4, 2023 at 1:31:28 PM
UTC+5:30 Dilawar Singh wrote:

The answer is not going to be simple.

Make sure that every time you save or open
the file, the application uses encoding
utf-8. Microsoft uses another weird
encoding called utf-16
(https://en.wikipedia.org/wiki/UTF-16).
Stay away from utf-16 it if you want to
copy-paste anything from the web or
web-based tools.

Long answer:

https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-develo

Re: [datameet] Creating variable names in Devanagari

[Nikhil VJ]

Hi Sharad,

If you have to import the data into a web-based backend, then your
programmers will be most comfortable with: GeoJSON format.
Because : It's a simple text/json format which can even be opened as raw
text, and so there are many different ways to load it in a program and use
data from it.
And this will also support Devnagri columns (which become property keys
here) same as geopackage.

Another thing to look out for: Ensure that the database being used by the
backend has the same UTF-8 encoding set.

--
Cheers,
Nikhil VJ
https://nikhilvj.co.in


On Mon, Dec 4, 2023 at 3:56 PM Sharad Lele  wrote:

> Thanks, Ujaval. Yes saving as .geopkg works (saves the field names in full
> and displays properly upon reopening). This is the only solution that has
> worked so far.
>
> But I am not sure this is going to work for us in this particular
> situation where we are then using the file in postGIS to display in a
> webGIS. Let me ask our webGIS person.
>
> Sharad
>
> On Monday, December 4, 2023 at 3:10:18 PM UTC+5:30 Ujaval Gandhi wrote:
>
>> Use the geopackage format. It's a much better modern format without the
>> limitations of shapefiles. The default encoding would be UTF-8 and it
>> should work. Tested with a new geopackage layer and it is able to create
>> column names and values in Devanagri script
>> https://i.imgur.com/2prELf6.png
>> 
>> ---
>> Ujaval Gandhi
>> Spatial Thoughts
>> www.spatialthoughts.com
>> 
>>
>>
>>
>> On Mon, Dec 4, 2023 at 2:36 PM Sharad Lele  wrote:
>>
>>> The shapefile encoding is set to UTF-8.  But any edits to variable names
>>> only last as long as I don't save the file. Save and reopening results in
>>> going back to the same truncated/funny display as above [image:
>>> Screenshot Layer properties.jpeg]
>>>
>>> On Monday, December 4, 2023 at 2:17:41 PM UTC+5:30 Dilawar Singh wrote:
>>>
 I found this
 https://gis.stackexchange.com/questions/381308/how-to-change-qgis-3-default-encoding.


 The file you are loading may have a different encoding that QGIS3. You
 have to verify this manually and make sure that QGIS3 and file encoding is
 the same.

 I am guessing the while QGIS3 is using utf-8, the file that you are
 loading may be utf-16 encoded. Not sure if there are automated tools that
 can convert UTF-16 files to UTF8 file while preserving unicode (
 https://superuser.com/questions/1786434/convert-utf-16-le-to-utf-8-in-windows-via-command-line
 ).

 best,
Dilawar



  On Mon, 04 Dec 2023 14:05:01 +0530 *Sharad Lele
 * wrote ---

 Just to clarify/muddy the situation: I am able to enter long Devanagari
 text in the District/Jila column without any problems. It is the
 column/field name itself that I run into problems, which may have to do
 with the 8 characteri limit?

 [image: Screenshot QGIS devanagari field names.jpeg]


 On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30 Sharad Lele wrote:

 Thanks, Dilawar. The article was very interesting/useful. But for a
 simpleton like me, the question then is: How do I ensure that QGIS3 in
 particular is using UTF-8 encoding every time?

 Sharad

 On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh wrote:

 The answer is not going to be simple.

 Make sure that every time you save or open the file, the application
 uses encoding utf-8. Microsoft uses another weird encoding called utf-16 (
 https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if
 you want to copy-paste anything from the web or web-based tools.

 Long answer:
 https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/

 best,
Dilawar



  On Mon, 04 Dec 2023 13:24:28 +0530 *Sharad Lele
 * wrote ---


 Hi folks, I am using QGIS and I want to create variable names in
 Devanagari (for instance जिला instead of District). If I create a new
 column in the attribute table and enter the column name in Devanagari using
 Microsoft Indic Language Tool on my computer, it seems to work (जिला is
 shown as the column name). But the moment I save the edits and reopen the
 attribute table, this gets truncated into something like जिल?. The last
 character is not really a question mark, something more complex.

 Any suggestions for making the column names appear and save properly in
 Devanagari? (The same would be true for Kannada or other scripts, I am
 sure).

>>

Re: [datameet] Creating variable names in Devanagari

[Sharad Lele]

Thanks, Ujaval. Yes saving as .geopkg works (saves the field names in full 
and displays properly upon reopening). This is the only solution that has 
worked so far.

But I am not sure this is going to work for us in this particular situation 
where we are then using the file in postGIS to display in a webGIS. Let me 
ask our webGIS person.

Sharad

On Monday, December 4, 2023 at 3:10:18 PM UTC+5:30 Ujaval Gandhi wrote:

> Use the geopackage format. It's a much better modern format without the 
> limitations of shapefiles. The default encoding would be UTF-8 and it 
> should work. Tested with a new geopackage layer and it is able to create 
> column names and values in Devanagri script 
> https://i.imgur.com/2prELf6.png 
> 
> ---
> Ujaval Gandhi
> Spatial Thoughts
> www.spatialthoughts.com 
> 
>
>
>
> On Mon, Dec 4, 2023 at 2:36 PM Sharad Lele  wrote:
>
>> The shapefile encoding is set to UTF-8.  But any edits to variable names 
>> only last as long as I don't save the file. Save and reopening results in 
>> going back to the same truncated/funny display as above [image: 
>> Screenshot Layer properties.jpeg]
>>
>> On Monday, December 4, 2023 at 2:17:41 PM UTC+5:30 Dilawar Singh wrote:
>>
>>> I found this 
>>> https://gis.stackexchange.com/questions/381308/how-to-change-qgis-3-default-encoding.
>>>  
>>>
>>>
>>> The file you are loading may have a different encoding that QGIS3. You 
>>> have to verify this manually and make sure that QGIS3 and file encoding is 
>>> the same. 
>>>
>>> I am guessing the while QGIS3 is using utf-8, the file that you are 
>>> loading may be utf-16 encoded. Not sure if there are automated tools that 
>>> can convert UTF-16 files to UTF8 file while preserving unicode (
>>> https://superuser.com/questions/1786434/convert-utf-16-le-to-utf-8-in-windows-via-command-line
>>> ).
>>>
>>> best,
>>>Dilawar 
>>>
>>>
>>>
>>>  On Mon, 04 Dec 2023 14:05:01 +0530 *Sharad Lele 
>>> * wrote ---
>>>
>>> Just to clarify/muddy the situation: I am able to enter long Devanagari 
>>> text in the District/Jila column without any problems. It is the 
>>> column/field name itself that I run into problems, which may have to do 
>>> with the 8 characteri limit?
>>>
>>> [image: Screenshot QGIS devanagari field names.jpeg]
>>>
>>>
>>> On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30 Sharad Lele wrote:
>>>
>>> Thanks, Dilawar. The article was very interesting/useful. But for a 
>>> simpleton like me, the question then is: How do I ensure that QGIS3 in 
>>> particular is using UTF-8 encoding every time?
>>>
>>> Sharad
>>>
>>> On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh wrote:
>>>
>>> The answer is not going to be simple. 
>>>
>>> Make sure that every time you save or open the file, the application 
>>> uses encoding utf-8. Microsoft uses another weird encoding called utf-16 (
>>> https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if you 
>>> want to copy-paste anything from the web or web-based tools.
>>>
>>> Long answer: 
>>> https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/
>>>
>>> best,
>>>Dilawar 
>>>
>>>
>>>
>>>  On Mon, 04 Dec 2023 13:24:28 +0530 *Sharad Lele 
>>> * wrote ---
>>>
>>>
>>> Hi folks, I am using QGIS and I want to create variable names in 
>>> Devanagari (for instance जिला instead of District). If I create a new 
>>> column in the attribute table and enter the column name in Devanagari using 
>>> Microsoft Indic Language Tool on my computer, it seems to work (जिला is 
>>> shown as the column name). But the moment I save the edits and reopen the 
>>> attribute table, this gets truncated into something like जिल?. The last 
>>> character is not really a question mark, something more complex. 
>>>
>>> Any suggestions for making the column names appear and save properly in 
>>> Devanagari? (The same would be true for Kannada or other scripts, I am 
>>> sure).
>>>
>>> Sharad
>>>
>>>
>>> --
>>> Datameet is a community of Data Science enthusiasts in India. Know more 
>>> about us by visiting http://datameet.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to datameet+u...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com
>>>  
>>> 
>>>

Re: [datameet] Creating variable names in Devanagari

[Dilawar Singh]

I found this 
https://gis.stackexchange.com/questions/381308/how-to-change-qgis-3-default-encoding.
 



The file you are loading may have a different encoding that QGIS3. You have to 
verify this manually and make sure that QGIS3 and file encoding is the same. 

I am guessing the while QGIS3 is using utf-8, the file that you are loading may 
be utf-16 encoded. Not sure if there are automated tools that can convert 
UTF-16 files to UTF8 file while preserving unicode 
(https://superuser.com/questions/1786434/convert-utf-16-le-to-utf-8-in-windows-via-command-line).


best,

   Dilawar 






 On Mon, 04 Dec 2023 14:05:01 +0530 Sharad Lele  
wrote ---



Just to clarify/muddy the situation: I am able to enter long Devanagari text in 
the District/Jila column without any problems. It is the column/field name 
itself that I run into problems, which may have to do with the 8 characteri 
limit?




On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30 Sharad Lele wrote:

Thanks, Dilawar. The article was very interesting/useful. But for a simpleton 
like me, the question then is: How do I ensure that QGIS3 in particular is 
using UTF-8 encoding every time?

Sharad


On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh wrote:

The answer is not going to be simple. 



Make sure that every time you save or open the file, the application uses 
encoding utf-8. Microsoft uses another weird encoding called utf-16 
(https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if you want 
to copy-paste anything from the web or web-based tools.



Long answer: 
https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/



best,

   Dilawar 







 On Mon, 04 Dec 2023 13:24:28 +0530 Sharad Lele <> wrote ---









Hi folks, I am using QGIS and I want to create variable names in Devanagari 
(for instance जिला instead of District). If I create a new column in the 
attribute table and enter the column name in Devanagari using Microsoft Indic 
Language Tool on my computer, it seems to work (जिला is shown as the column 
name). But the moment I save the edits and reopen the attribute table, this 
gets truncated into something like जिल?. The last character is not really a 
question mark, something more complex. 



Any suggestions for making the column names appear and save properly in 
Devanagari? (The same would be true for Kannada or other scripts, I am sure).



Sharad








--

Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org

--- 

You received this message because you are subscribed to the Google Groups 
"datameet" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to .

To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com?utm_medium=email&utm_source=footer.















--
 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google Groups 
"datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/29ffbab3-117a-4a67-8988-ae253db7e2cbn%40googlegroups.com?utm_medium=email&utm_source=footer.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/18c3402b8ae.6e57e54a1273512.3635000473182355283%40gmail.com.

Re: [datameet] Creating variable names in Devanagari

[Sharad Lele]

Just to clarify/muddy the situation: I am able to enter long Devanagari 
text in the District/Jila column without any problems. It is the 
column/field name itself that I run into problems, which may have to do 
with the 8 characteri limit?

[image: Screenshot QGIS devanagari field names.jpeg]


On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30 Sharad Lele wrote:

> Thanks, Dilawar. The article was very interesting/useful. But for a 
> simpleton like me, the question then is: How do I ensure that QGIS3 in 
> particular is using UTF-8 encoding every time?
>
> Sharad
>
> On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh wrote:
>
>> The answer is not going to be simple. 
>>
>> Make sure that every time you save or open the file, the application uses 
>> encoding utf-8. Microsoft uses another weird encoding called utf-16 (
>> https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if you 
>> want to copy-paste anything from the web or web-based tools.
>>
>> Long answer: 
>> https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/
>>
>> best,
>>Dilawar 
>>
>>
>>
>>  On Mon, 04 Dec 2023 13:24:28 +0530 *Sharad Lele 
>> * wrote ---
>>
>> Hi folks, I am using QGIS and I want to create variable names in 
>> Devanagari (for instance जिला instead of District). If I create a new 
>> column in the attribute table and enter the column name in Devanagari using 
>> Microsoft Indic Language Tool on my computer, it seems to work (जिला is 
>> shown as the column name). But the moment I save the edits and reopen the 
>> attribute table, this gets truncated into something like जिल?. The last 
>> character is not really a question mark, something more complex. 
>>
>> Any suggestions for making the column names appear and save properly in 
>> Devanagari? (The same would be true for Kannada or other scripts, I am 
>> sure).
>>
>> Sharad
>>
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to datameet+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com
>>  
>> 
>> .
>>
>>
>>
>>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/29ffbab3-117a-4a67-8988-ae253db7e2cbn%40googlegroups.com.

Re: [datameet] Creating variable names in Devanagari

[Sharad Lele]

Thanks, Dilawar. The article was very interesting/useful. But for a 
simpleton like me, the question then is: How do I ensure that QGIS3 in 
particular is using UTF-8 encoding every time?

Sharad

On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh wrote:

> The answer is not going to be simple. 
>
> Make sure that every time you save or open the file, the application uses 
> encoding utf-8. Microsoft uses another weird encoding called utf-16 (
> https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if you 
> want to copy-paste anything from the web or web-based tools.
>
> Long answer: 
> https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/
>
> best,
>Dilawar 
>
>
>
>  On Mon, 04 Dec 2023 13:24:28 +0530 *Sharad Lele * 
> wrote ---
>
> Hi folks, I am using QGIS and I want to create variable names in 
> Devanagari (for instance जिला instead of District). If I create a new 
> column in the attribute table and enter the column name in Devanagari using 
> Microsoft Indic Language Tool on my computer, it seems to work (जिला is 
> shown as the column name). But the moment I save the edits and reopen the 
> attribute table, this gets truncated into something like जिल?. The last 
> character is not really a question mark, something more complex. 
>
> Any suggestions for making the column names appear and save properly in 
> Devanagari? (The same would be true for Kannada or other scripts, I am 
> sure).
>
> Sharad
>
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more 
> about us by visiting http://datameet.org
> --- 
> You received this message because you are subscribed to the Google Groups 
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to datameet+u...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com
>  
> 
> .
>
>
>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/feff88da-2d89-4f0b-aa7b-4342f78fac2dn%40googlegroups.com.

Re: [datameet] Creating variable names in Devanagari

[Dilawar Singh]

The answer is not going to be simple. 



Make sure that every time you save or open the file, the application uses 
encoding utf-8. Microsoft uses another weird encoding called utf-16 
(https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if you want 
to copy-paste anything from the web or web-based tools.



Long answer: 
https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/



best,

   Dilawar 






 On Mon, 04 Dec 2023 13:24:28 +0530 Sharad Lele  
wrote ---



Hi folks, I am using QGIS and I want to create variable names in Devanagari 
(for instance जिला instead of District). If I create a new column in the 
attribute table and enter the column name in Devanagari using Microsoft Indic 
Language Tool on my computer, it seems to work (जिला is shown as the column 
name). But the moment I save the edits and reopen the attribute table, this 
gets truncated into something like जिल?. The last character is not really a 
question mark, something more complex. 



Any suggestions for making the column names appear and save properly in 
Devanagari? (The same would be true for Kannada or other scripts, I am sure).



Sharad



--

 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org

 --- 

 You received this message because you are subscribed to the Google Groups 
"datameet" group.

 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.

 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com?utm_medium=email&utm_source=footer.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/18c33d876d0.3df668871267863.431381660657550485%40gmail.com.

[datameet] Creating variable names in Devanagari

[Sharad Lele]

Hi folks, I am using QGIS and I want to create variable names in Devanagari 
(for instance जिला instead of District). If I create a new column in the 
attribute table and enter the column name in Devanagari using Microsoft 
Indic Language Tool on my computer, it seems to work (जिला is shown as the 
column name). But the moment I save the edits and reopen the attribute 
table, this gets truncated into something like जिल?. The last character is 
not really a question mark, something more complex. 

Any suggestions for making the column names appear and save properly in 
Devanagari? (The same would be true for Kannada or other scripts, I am 
sure).

Sharad

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com.