Re: Why doesn't Django support managing Raw SQL changes using migration system like models

2018-02-28 Thread Julio Biason 
Hi Cmp,

Well, Django has to be database agnostic for things to work with any
database. If raw sql is allowed, how do you can be sure it will work with
any database?

So far, you mentioned a bunch of PostreSQL things. But what if, tomorrow,
some higher ups decided that support is required and buy Oracle (because
higher ups usually aren't that smart). How things will work? How do you
make sure your raw SQL commands will work? (IIRC, there are differences in
SQL/DML between Oracle and MySQL).

The biggest problem I see is that you're using triggers. Django knows
nothing about them and since you're giving the database control to Django
(via its migrations and querysets), you should relegate all the things to
Django. And yes, there are drawbacks on that too, but still, everything
should be one single place -- and if you need other systems to add data to
your database, you should provide them with an API instead of letting them
add directly to the database.

The biggest problem I see on your solution, is that your giving powers over
your data to two different things -- Django and the database via triggers
-- and that's the first step into mess.

PS: 76 migrations is still nothing. I worked with databases with 150+
migrations, and nobody cared about what the migrations did, because the
models -- in their current, up-to-date form -- is the canonical
representation of the data. All those 150+ migrations were just the way to
reach the current state.

On Tue, Feb 27, 2018 at 8:53 PM, cmp  wrote:

> I have an problem that I wanted to solve recently using a Postgres Trigger
> + Procedure that I do not want to do via Signals or within Python itself.
>
> I want to make a log which when an entry is created the result is applied
> to a row. We need the log to be the source of truth and I don't want to
> write a whole async system using something like Kafka or whatever everyone
> recommends.
>
> The issue here is that I need to iterate on the procedure which updates
> the rows as features come along and make it easy for other developers to
> find/read/etc.
>
> Django only allows storing things like this as RawSql migration commands
> inside the migrations folder but this App already has 76 migrations and I
> think it's probably unreasonable for me to assume a normal django developer
> will think to read all those migrations without running in to some problem
> with my triggers. Not to mention the main block of sql will need to be copy
> and pasted between changes to new migrations... etc.
>
> This seems like something the migration system should support somehow.
>
> I found a package which seems to do this, but it appears unmaintained
> (little to no activity): https://github.com/klichukb/django-migrate-sql
>
> I think it'd be a nice feature for Django to support maintaining a set of
> Raw SQL via the migration system so more advanced features of databases can
> be used.
>
> For instance in Postgres setting up other things like full text search and
> special search indexes were difficult in 1.8. I realize it's easier since
> 1.10 to add a full text search index, but adding that feature is much more
> difficult than the RawSQL migrations we did to implement adding the index
> before.
>
> Making it easy to manage triggers and procedures via the migration system
> is very difficult, but I think supporting RawSQL items which are managed
> via migrations would be relatively easy to implement and add a lot of value.
>
> I'm posting this for two reasons: 1) I think it's a good idea and 2) to
> let others show me where I'm missing something or why this wouldn't be a
> good fit for Django.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/8043d044-3f7c-4bd2-bdc4-c504faf5120b%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*Julio Biason*, Sofware Engineer
*AZION*  |  Deliver. Accelerate. Protect.
Office: +55 51 3083 8101   |  Mobile: +55 51
*99907 0554*

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEM7gE3k2ugP0X7RwJwOrxrKB-LQZ48uQ%3DMjbsF

Why doesn't Django support managing Raw SQL changes using migration system like models

2018-02-27 Thread cmp 
I have an problem that I wanted to solve recently using a Postgres Trigger 
+ Procedure that I do not want to do via Signals or within Python itself.

I want to make a log which when an entry is created the result is applied 
to a row. We need the log to be the source of truth and I don't want to 
write a whole async system using something like Kafka or whatever everyone 
recommends.

The issue here is that I need to iterate on the procedure which updates the 
rows as features come along and make it easy for other developers to 
find/read/etc.

Django only allows storing things like this as RawSql migration commands 
inside the migrations folder but this App already has 76 migrations and I 
think it's probably unreasonable for me to assume a normal django developer 
will think to read all those migrations without running in to some problem 
with my triggers. Not to mention the main block of sql will need to be copy 
and pasted between changes to new migrations... etc.

This seems like something the migration system should support somehow.

I found a package which seems to do this, but it appears unmaintained 
(little to no activity): https://github.com/klichukb/django-migrate-sql

I think it'd be a nice feature for Django to support maintaining a set of 
Raw SQL via the migration system so more advanced features of databases can 
be used.

For instance in Postgres setting up other things like full text search and 
special search indexes were difficult in 1.8. I realize it's easier since 
1.10 to add a full text search index, but adding that feature is much more 
difficult than the RawSQL migrations we did to implement adding the index 
before.

Making it easy to manage triggers and procedures via the migration system 
is very difficult, but I think supporting RawSQL items which are managed 
via migrations would be relatively easy to implement and add a lot of value.

I'm posting this for two reasons: 1) I think it's a good idea and 2) to let 
others show me where I'm missing something or why this wouldn't be a good 
fit for Django.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/8043d044-3f7c-4bd2-bdc4-c504faf5120b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.