Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
Hi Robert, Thanks again for debugging this issue. I can confirm that the osm2pgsql segfault is fixed with a rebuilt libprotobuf-c1 that includes your patch. Shall I reassign this issue to libprotobuf-c so you can close it with the upload of the new upstream release when it's out? Kind Regards, Bas -- GPG Key ID: 4096R/E88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
Sebastiaan Couwenberg wrote: Hi Robert, Thanks again for debugging this issue. I can confirm that the osm2pgsql segfault is fixed with a rebuilt libprotobuf-c1 that includes your patch. Great, thanks for confirming. Shall I reassign this issue to libprotobuf-c so you can close it with the upload of the new upstream release when it's out? Yes, please. -- Robert Edmonds edmo...@debian.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
Package: osm2pgsql Version: 0.84.0-2 Severity: important When using osm2pgsql 0.84.0-2, I get the following error when importing data from a pbf file. If I downgrade to 0.84.0-1, it works as expected. Processing: Node(20k 20.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)*** Error in `osm2pgsql': free(): invalid pointer: 0x7f4613a62678 *** I am using Debian Sid. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
Hi Christopher, This is likely related to the changes for the protobuf-c transition (#73). Can you verify that osm2pgsql 0.84.0-2 doesn't have this problem importing data from a OSM XML file instead of PBF? It would also have been helpful if you had used reportbug(1) to report this issue so that we know which version of the dependencies you have installed. Kind Regards, Bas -- GPG Key ID: 4096R/E88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
Package: osm2pgsql Version: 0.84.0-2 Followup-For: Bug #756983 Control: tags -1 confirmed I can confirm the segfault while importing a PBF file with osm2pgsql. osm2pgsql 0.84.0-1 depends on libprotobuf-c0 (= 0.10), whereas 0.84.0-2 depends on libprotobuf-c1 (= 1.0.0-1~). osm2pgsql upstream uses protobuf_c_default_allocator instead of NULL as suggested by Robert Edmonds. This looks like the cause of the problem. Kind Regards, Bas -- System Information: Debian Release: jessie/sid APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.14-2-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages osm2pgsql depends on: ii libbz2-1.0 1.0.6-7 ii libc6 2.19-7 ii libgcc1 1:4.9.1-4 ii libgeos-3.4.2 3.4.2-5 ii liblua5.2-0 5.2.3-1 ii libpq5 9.4~beta2-1 ii libproj04.8.0-5 ii libprotobuf-c1 1.0.0-1 ii libstdc++6 4.9.1-4 ii libxml2 2.9.1+dfsg1-4 ii zlib1g 1:1.2.8.dfsg-1 Versions of packages osm2pgsql recommends: ii postgis 2.1.3+dfsg-3 ii postgresql-9.3-postgis-2.1 2.1.3+dfsg-3 osm2pgsql suggests no packages. -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
Bas Couwenberg wrote: I can confirm the segfault while importing a PBF file with osm2pgsql. osm2pgsql 0.84.0-1 depends on libprotobuf-c0 (= 0.10), whereas 0.84.0-2 depends on libprotobuf-c1 (= 1.0.0-1~). osm2pgsql upstream uses protobuf_c_default_allocator instead of NULL as suggested by Robert Edmonds. This looks like the cause of the problem. Hi, I'd be happy to take a look and try to debug the issue to see if it's caused by the protobuf-c changes. However, I'm entirely unfamiliar with osm2pgsql. Is there any chance you could walk me through setting up a test case, or provide a sample file and command-line arguments that exhibit the problem? Thanks! -- Robert Edmonds edmo...@debian.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
On 08/04/2014 05:59 PM, Robert Edmonds wrote: I'd be happy to take a look and try to debug the issue to see if it's caused by the protobuf-c changes. Thanks, that's much appreciated. However, I'm entirely unfamiliar with osm2pgsql. Is there any chance you could walk me through setting up a test case, or provide a sample file and command-line arguments that exhibit the problem? The basic setup is described in the Operation section of the README file (/usr/share/doc/osm2pgsql/README.gz). For my own test case I used a wheezy based database server: createdb osm createlang plpgsql osm psql osm /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql psql osm /usr/share/postgresql/9.1/contrib/postgis-1.5/spatial_ref_sys.sql psql -d osm -c ALTER TABLE geometry_columns OWNER TO bas psql -d osm -c ALTER TABLE spatial_ref_sys OWNER TO bas psql osm ~/git/pkg-grass/osm2pgsql/900913.sql echo CREATE EXTENSION hstore | psql osm Adjust the paths for PostgreSQL 9.3 on sid. OSM PBF files can be downloaded from Geofabrik: http://download.geofabrik.de/ I used an old one I had lying around. osm2pgsql command line I used: osm2pgsql -c -l -s -d dbname -U dbuser -W -H dbhost -v netherlands-140201.osm.pbf Thanks! And thank you too. Kind Regards, Bas -- GPG Key ID: 4096R/E88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#756983: Error in `osm2pgsql': free(): invalid pointer: 0x00007f4613a62678 ***
Sebastiaan Couwenberg wrote:
On 08/04/2014 05:59 PM, Robert Edmonds wrote:
I'd be happy to take a look and try to debug the issue to see if it's
caused by the protobuf-c changes.
Thanks, that's much appreciated.
However, I'm entirely unfamiliar with osm2pgsql. Is there any chance
you could walk me through setting up a test case, or provide a sample
file and command-line arguments that exhibit the problem?
The basic setup is described in the Operation section of the README file
(/usr/share/doc/osm2pgsql/README.gz).
For my own test case I used a wheezy based database server:
createdb osm
createlang plpgsql osm
psql osm /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql
psql osm
/usr/share/postgresql/9.1/contrib/postgis-1.5/spatial_ref_sys.sql
psql -d osm -c ALTER TABLE geometry_columns OWNER TO bas
psql -d osm -c ALTER TABLE spatial_ref_sys OWNER TO bas
psql osm ~/git/pkg-grass/osm2pgsql/900913.sql
echo CREATE EXTENSION hstore | psql osm
Adjust the paths for PostgreSQL 9.3 on sid.
OSM PBF files can be downloaded from Geofabrik:
http://download.geofabrik.de/
I used an old one I had lying around.
osm2pgsql command line I used:
osm2pgsql -c -l -s -d dbname -U dbuser -W -H dbhost -v
netherlands-140201.osm.pbf
Thanks, I was able to figure out how to get an osm2pgsql setup working
with the above instructions and get a reproducible test case.
It looks like we (protobuf-c) were not explicitly zero'ing out memory in
certain cases, namely, when unpacking a 0-length byte string field in a
protobuf message. (In the .osm.pbf file that I tried, this case
reliably appeared in the first element of the StringTable field in
PrimitiveBlock messages.) This was causing a stray pointer to
occasionally be passed to free() when parse-pbf.c's processOsmData()
called primitive_block__free_unpacked(). Most of the time the affected
ProtobufCBinaryData object happens to be allocated with zero'd memory by
chance, which is why this bug wasn't caught sooner.
Here is the patch to protobuf-c. I'll be releasing a new upstream
version of protobuf-c with this fix soon.
diff --git a/protobuf-c/protobuf-c.c b/protobuf-c/protobuf-c.c
index c7fb21d..b66ed93 100644
--- a/protobuf-c/protobuf-c.c
+++ b/protobuf-c/protobuf-c.c
@@ -2304,6 +2304,8 @@ parse_required_member(ScannedMember *scanned_member,
if (bd-data == NULL)
return FALSE;
memcpy(bd-data, data + pref_len, len - pref_len);
+ } else {
+ bd-data = NULL;
}
bd-len = len - pref_len;
return TRUE;
--
Robert Edmonds
edmo...@debian.org
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
