You should create a symfony task, usable this way : ./symfony cities-csv-import path/to/file
This command would parse the Csv and use propel model to create new entries. Consider chunk import if your file is very very big. For example : ./symfony cities...... --from 0 --to 5000 file.Csv Put your big file on a correct file storage system (samba, ftp...), so your subversioned project keep a correct size. Put a sample Csv in project for testing, add some fixtures (yml) for development. At the end, in documentation of project, explain how to use command, where to find the big file, and any useful information. Imho. Alex' --- Alexandre Salomé - http://alexandre-salome.fr Le 20 nov. 2010 15:14, "Felix E. Klee" <felix.k...@inka.de> a écrit : For an application I am maintaining, there are various data sets that should get uploaded into the MySQL database with each deployment, and of course repeatedly during development. For example there is a data set for cities, which is quite large. The data is available as CSV files, which are structured *differently* than the MySQL database. The plan: Write a command line script that converts CSV files into SQL. Then upload the SQL into the database. What I also considered: * Conversion to fixtures instead of SQL: That may make things a little bit more database-independent. The disadvantage is that fixtures don't seem to be made for big data sets. Their name suggests that they are made for testing. * Instead of using a command line script, allow uploading or maintenance of the data via an admin web interface. This is not necessary for now, and we would still need to have a mechanism that makes deployment of data from staging to production simple. Anyway, I'd like to know: * Where should the CSV and the (temporary) SQL files be stored in order to adhere to Symfony's conventions? * Where should I put the conversion script? * How would I best integrate into the Symfony project: conversion from CSV and uploading of the SQL data? We use: Symfony Version 1.1 (to be updated to 1.4) with Propel -- Dipl.-Phys. Felix E. Klee Mobile: +34.693759173, +49.174.1386060 Fax: +49.3212.1021049 -- If you want to report a vulnerability issue on symfony, please send it to security at symfony-project.com You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to symfony-users@googlegroups.com To unsubscribe from this group, send email to symfony-users+unsubscr...@googlegroups.com<symfony-users%2bunsubscr...@googlegroups.com> For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en -- If you want to report a vulnerability issue on symfony, please send it to security at symfony-project.com You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to symfony-users@googlegroups.com To unsubscribe from this group, send email to symfony-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en
