I use the attached patch, which tweaks shp2pgsql to handle newlines
(\n) and carriage returns (\r), replaces hi-bit characters with a
question mark, and removes some geometry checks.
You probably only want to do the first of those three. In that case,
the patch should be easy to modify. Hope you find it helpful.
-Dave
On Fri, Apr 3, 2009 at 9:21 AM, Ivan Mincik <[email protected]> wrote:
> On Friday 03 April 2009, Stefan Schwarzer wrote:
>> Hi there,
>>
>> I have a shapefile I would like to import into my database, but the
>> shp2pgsql seems to have problems with carriage returns in the DBF:
>>
>> ERROR: literal carriage return found in data
>> HINT: Use "\r" to represent carriage return.
>>
>> What can I do about it? I don't see any solution, other than deleting
>> the column where this occurs. But that's not very cool...
>>
>> Thanks for any hints,
>>
>> Stef
> Try to do not use dump format ( don't use parameter -D). shp2pgsql will use
> simple INSERT INTO command. It will take longer time to import, but it can
> solve your problem.
> Ivan
>>
>> _______________________________________________
>> postgis-users mailing list
>> [email protected]
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>
>
> _______________________________________________
> postgis-users mailing list
> [email protected]
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
Index: shp2pgsql.c
===================================================================
--- shp2pgsql.c (revision 3479)
+++ shp2pgsql.c (working copy)
@@ -158,8 +158,9 @@
/*
* find all the tabs and make them \<tab>s
*
- * 1. find # of tabs
- * 2. make new string
+ * 1. replace hi-bit characters with '?'
+ * 2. find # of tabs, carriage returns, and newlines
+ * 3. make new string
*
* we dont escape already escaped tabs
*/
@@ -178,11 +179,18 @@
str = utf8str;
}
#endif
+
+ // replace hi-bit characters
+ ptr=str;
+ while (*ptr) {
+ if (*ptr < 0) *ptr = '?';
+ ++ptr;
+ }
ptr = str;
while (*ptr) {
- if ( *ptr == '\t' || *ptr == '\\' ) toescape++;
+ if ( *ptr == '\t' || *ptr == '\\' || *ptr == '\r' || *ptr == '\n' ) toescape++;
ptr++;
}
@@ -195,8 +203,19 @@
optr=result;
ptr=str;
while (*ptr) {
- if ( *ptr == '\t' || *ptr == '\\' ) *optr++='\\';
- *optr++=*ptr++;
+ if ( *ptr == '\r' )
+ {
+ *optr++='\\';
+ *optr++='r';
+ ptr++;
+ } else if ( *ptr == '\n' ) {
+ *optr++='\\';
+ *optr++='n';
+ ptr++;
+ } else {
+ if ( *ptr == '\t' || *ptr == '\\') *optr++='\\';
+ *optr++=*ptr++;
+ }
}
*optr='\0';
@@ -215,8 +234,9 @@
* find all quotes and make them \quotes
* find all '\' and make them '\\'
*
- * 1. find # of characters
- * 2. make new string
+ * 1. Replace hi-bit characters
+ * 2. find # of characters
+ * 3. make new string
*/
char *result;
@@ -233,6 +253,13 @@
str = utf8str;
}
#endif
+
+ // replace hi-bit characters
+ ptr=str;
+ while (*ptr) {
+ if (*ptr < 0) *ptr = '?';
+ ++ptr;
+ }
ptr = str;
@@ -428,6 +455,9 @@
*/
GetFieldsSpec();
+ // dfuhry
+ printf("SET client_min_messages TO WARNING;\n");
+
printf("BEGIN;\n");
/*
@@ -692,7 +722,7 @@
continue;
}
}
-
+
if (!dump_format)
{
if ( schema )
@@ -1172,12 +1202,13 @@
}
if (!hwgeom)
- result = serialized_lwgeom_to_hexwkb(&lwg_unparser_result, serialized_lwgeom, PARSER_CHECK_ALL, -1);
+ result = serialized_lwgeom_to_hexwkb(&lwg_unparser_result, serialized_lwgeom, PARSER_CHECK_NONE, -1);
else
result = serialized_lwgeom_to_ewkt(&lwg_unparser_result, serialized_lwgeom, PARSER_CHECK_ALL);
if (result)
{
+ fprintf(stderr, "lwg_unparser error:\n");
fprintf(stderr, "ERROR: %s\n", lwg_unparser_result.message);
exit(1);
}
_______________________________________________
postgis-users mailing list
[email protected]
http://postgis.refractions.net/mailman/listinfo/postgis-users
