from:"Kevin Yu"

Re: Would old version of mysql caused performance issue in Django

2017-07-18 Thread Kevin Yu

I'm not trying to update database 5.1 to 5.7. I am simplying seeing 
difference in terms of performance when running against different version 
of Mysql with same dataset.

Can you please explain how dumpdata could help?

On Tuesday, July 18, 2017 at 12:46:43 PM UTC-7, sacrac wrote:
>
> why you try update the databases 5.1 to 5.7?
>
> try  dumpdate with manage.py django in .json all apps y then loaddata in 
> new database!
>
> https://docs.djangoproject.com/en/1.11/ref/django-admin/#dumpdata
>
> cheers
>
> On Tue, Jul 18, 2017 at 1:12 PM, Felipe Brunelli de Andrade <
> fbrua...@gmail.com > wrote:
>
>> It seems like some thing in your app view.
>>
>> Did you try to dump real database and run localy in your workstation?
>>
>> 2017-07-18 16:05 GMT-03:00 Kevin Yu <sams...@gmail.com >:
>>
>>>
>>> <https://lh3.googleusercontent.com/-jIXPWhwHNUw/WW5biPCMzlI/De4/BnmWrwpKd88NFW-PswfbXcBNZSN04c36ACLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B12-03-18.png>
>>>
>>> I even noticed when i do runserver on my local to connect to the older 
>>> version of mysql db, the starting of dev server is noticeably slower.
>>>
>>> What I mean is here:
>>> kevin@kevin-OptiPlex-7020:~/repo/Infosite_Django/app$ python manage.py 
>>> runserver
>>> Performing system checks...
>>>
>>> System check identified no issues (0 silenced).
>>> July 18, 2017 - 19:00:12
>>> Django version 1.11.2, using settings 'infosite.settings'
>>> Starting development server at http://127.0.0.1:8000/
>>> Quit the server with CONTROL-C.
>>>
>>>
>>> On Tuesday, July 18, 2017 at 11:43:53 AM UTC-7, Felipe wrote:
>>>>
>>>> Try to debug in chrome just press F12 and go to network tab and see 
>>>> the times fot load content
>>>>
>>>> 2017-07-18 15:18 GMT-03:00 Kevin Yu <sams...@gmail.com>:
>>>>
>>>>> I've eliminated the possibility of mysql performance issue by directly 
>>>>> running the query on the database. It's not network issue either since my 
>>>>> ping is just fine.
>>>>>
>>>>>
>>>>> On Tuesday, July 18, 2017 at 11:00:58 AM UTC-7, Kevin Yu wrote:
>>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I'm rewritting an web application using Django 1.11. When i hooked up 
>>>>>> to my test mysql database (version 5.7), the performance is amazing. The 
>>>>>> page renders within 1 second. However, when I connect to the existing 
>>>>>> production mysql (version 5.1), the page takes more than 10 seconds. 
>>>>>>
>>>>>> I installed the debug toolbar, and I found out the the query time is 
>>>>>> actually not the issue. Most of the time are in the CPU. 
>>>>>>
>>>>>> I am wondering whether the Django is having issues with older mysql 
>>>>>> version when Django receives the data and try to map it to objects via 
>>>>>> ORM.
>>>>>>
>>>>>>
>>>>>> <https://lh3.googleusercontent.com/-lti9EnCOwpI/WW5My639ZTI/Des/gUJgzX0Wy5IvrGpf2xGDKFbAkoqN7za2gCLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B10-59-30.png>
>>>>>>
>>>>>> -- 
>>>>> You received this message because you are subscribed to the Google 
>>>>> Groups "Django users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>>> an email to django-users...@googlegroups.com.
>>>>> To post to this group, send email to django...@googlegroups.com.
>>>>> Visit this group at https://groups.google.com/group/django-users.
>>>>> To view this discussion on the web visit 
>>>>> https://groups.google.com/d/msgid/django-users/ac4201dd-5267-4566-8d0a-82b2b2f043c4%40googlegroups.com
>>>>>  
>>>>> <https://groups.google.com/d/msgid/django-users/ac4201dd-5267-4566-8d0a-82b2b2f043c4%40googlegroups.com?utm_medium=email_source=footer>
>>>>> .
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-users.

Re: Would old version of mysql caused performance issue in Django

2017-07-18 Thread Kevin Yu

Yes, my local database is actually a mysqldump of the production so in
terms of data set, it's the same

On Jul 18, 2017 12:13 PM, "Felipe Brunelli de Andrade" <
fbruandr...@gmail.com> wrote:

It seems like some thing in your app view.

Did you try to dump real database and run localy in your workstation?

2017-07-18 16:05 GMT-03:00 Kevin Yu <samsun...@gmail.com>:

>
> <https://lh3.googleusercontent.com/-jIXPWhwHNUw/WW5biPCMzlI/De4/BnmWrwpKd88NFW-PswfbXcBNZSN04c36ACLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B12-03-18.png>
>
> I even noticed when i do runserver on my local to connect to the older
> version of mysql db, the starting of dev server is noticeably slower.
>
> What I mean is here:
> kevin@kevin-OptiPlex-7020:~/repo/Infosite_Django/app$ python manage.py
> runserver
> Performing system checks...
>
> System check identified no issues (0 silenced).
> July 18, 2017 - 19:00:12
> Django version 1.11.2, using settings 'infosite.settings'
> Starting development server at http://127.0.0.1:8000/
> Quit the server with CONTROL-C.
>
>
> On Tuesday, July 18, 2017 at 11:43:53 AM UTC-7, Felipe wrote:
>>
>> Try to debug in chrome just press F12 and go to network tab and see
>> the times fot load content
>>
>> 2017-07-18 15:18 GMT-03:00 Kevin Yu <sams...@gmail.com>:
>>
>>> I've eliminated the possibility of mysql performance issue by directly
>>> running the query on the database. It's not network issue either since my
>>> ping is just fine.
>>>
>>>
>>> On Tuesday, July 18, 2017 at 11:00:58 AM UTC-7, Kevin Yu wrote:
>>>>
>>>> Hi All,
>>>>
>>>> I'm rewritting an web application using Django 1.11. When i hooked up
>>>> to my test mysql database (version 5.7), the performance is amazing. The
>>>> page renders within 1 second. However, when I connect to the existing
>>>> production mysql (version 5.1), the page takes more than 10 seconds.
>>>>
>>>> I installed the debug toolbar, and I found out the the query time is
>>>> actually not the issue. Most of the time are in the CPU.
>>>>
>>>> I am wondering whether the Django is having issues with older mysql
>>>> version when Django receives the data and try to map it to objects via ORM.
>>>>
>>>>
>>>> <https://lh3.googleusercontent.com/-lti9EnCOwpI/WW5My639ZTI/Des/gUJgzX0Wy5IvrGpf2xGDKFbAkoqN7za2gCLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B10-59-30.png>
>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users...@googlegroups.com.
>>> To post to this group, send email to django...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-users.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/django-users/ac4201dd-5267-4566-8d0a-82b2b2f043c4%40googlegroups.com
>>> <https://groups.google.com/d/msgid/django-users/ac4201dd-5267-4566-8d0a-82b2b2f043c4%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
>
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-users/9efab471-ae94-4ff2-a62c-67650638c4e0%40googlegroups.com
> <https://groups.google.com/d/msgid/django-users/9efab471-ae94-4ff2-a62c-67650638c4e0%40googlegroups.com?utm_medium=email_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to a topic in the
Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/django-users/_EOK0u_We5k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/

Re: Would old version of mysql caused performance issue in Django

2017-07-18 Thread Kevin Yu



<https://lh3.googleusercontent.com/-jIXPWhwHNUw/WW5biPCMzlI/De4/BnmWrwpKd88NFW-PswfbXcBNZSN04c36ACLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B12-03-18.png>

I even noticed when i do runserver on my local to connect to the older 
version of mysql db, the starting of dev server is noticeably slower.

What I mean is here:
kevin@kevin-OptiPlex-7020:~/repo/Infosite_Django/app$ python manage.py 
runserver
Performing system checks...

System check identified no issues (0 silenced).
July 18, 2017 - 19:00:12
Django version 1.11.2, using settings 'infosite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.


On Tuesday, July 18, 2017 at 11:43:53 AM UTC-7, Felipe wrote:
>
> Try to debug in chrome just press F12 and go to network tab and see 
> the times fot load content
>
> 2017-07-18 15:18 GMT-03:00 Kevin Yu <sams...@gmail.com >:
>
>> I've eliminated the possibility of mysql performance issue by directly 
>> running the query on the database. It's not network issue either since my 
>> ping is just fine.
>>
>>
>> On Tuesday, July 18, 2017 at 11:00:58 AM UTC-7, Kevin Yu wrote:
>>>
>>> Hi All,
>>>
>>> I'm rewritting an web application using Django 1.11. When i hooked up to 
>>> my test mysql database (version 5.7), the performance is amazing. The page 
>>> renders within 1 second. However, when I connect to the existing production 
>>> mysql (version 5.1), the page takes more than 10 seconds. 
>>>
>>> I installed the debug toolbar, and I found out the the query time is 
>>> actually not the issue. Most of the time are in the CPU. 
>>>
>>> I am wondering whether the Django is having issues with older mysql 
>>> version when Django receives the data and try to map it to objects via ORM.
>>>
>>>
>>> <https://lh3.googleusercontent.com/-lti9EnCOwpI/WW5My639ZTI/Des/gUJgzX0Wy5IvrGpf2xGDKFbAkoqN7za2gCLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B10-59-30.png>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/ac4201dd-5267-4566-8d0a-82b2b2f043c4%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-users/ac4201dd-5267-4566-8d0a-82b2b2f043c4%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/9efab471-ae94-4ff2-a62c-67650638c4e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Would old version of mysql caused performance issue in Django

2017-07-18 Thread Kevin Yu



I've eliminated the possibility of mysql performance issue by directly 
running the query on the database. It's not network issue either since my 
ping is just fine.


On Tuesday, July 18, 2017 at 11:00:58 AM UTC-7, Kevin Yu wrote:
>
> Hi All,
>
> I'm rewritting an web application using Django 1.11. When i hooked up to 
> my test mysql database (version 5.7), the performance is amazing. The page 
> renders within 1 second. However, when I connect to the existing production 
> mysql (version 5.1), the page takes more than 10 seconds. 
>
> I installed the debug toolbar, and I found out the the query time is 
> actually not the issue. Most of the time are in the CPU. 
>
> I am wondering whether the Django is having issues with older mysql 
> version when Django receives the data and try to map it to objects via ORM.
>
>
> <https://lh3.googleusercontent.com/-lti9EnCOwpI/WW5My639ZTI/Des/gUJgzX0Wy5IvrGpf2xGDKFbAkoqN7za2gCLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B10-59-30.png>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ac4201dd-5267-4566-8d0a-82b2b2f043c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Would old version of mysql caused performance issue in Django

2017-07-18 Thread Kevin Yu

Hi Felipe,

Thanks for the quick response. I've eliminated it's cpu load on the 
production server by directing running the query on the mysql and 
calculation time is very quick. It's unlikely network issue because the 
ping is pretty quick.

mysql query time on the server:
1063 rows in set (0.00 sec)


ping 172.16.100.131
64 bytes from 172.16.100.131: icmp_seq=2 ttl=62 time=0.316 ms


On Tuesday, July 18, 2017 at 11:10:40 AM UTC-7, Felipe wrote:
>
> You need to see de CPU load in server.
>
> In your test you probably are using a local machine with no loading... but 
> your produce server I dont know. You need to see in server the webserver 
> too, try to run django-admin runserver :8000 and see if you have the 
> same issue.
>
> But if you are just connecting to another machine with database, and 
> application is in yur development enviroment, probably is your network.
>
> 2017-07-18 15:00 GMT-03:00 Kevin Yu <sams...@gmail.com >:
>
>> Hi All,
>>
>> I'm rewritting an web application using Django 1.11. When i hooked up to 
>> my test mysql database (version 5.7), the performance is amazing. The page 
>> renders within 1 second. However, when I connect to the existing production 
>> mysql (version 5.1), the page takes more than 10 seconds. 
>>
>> I installed the debug toolbar, and I found out the the query time is 
>> actually not the issue. Most of the time are in the CPU. 
>>
>> I am wondering whether the Django is having issues with older mysql 
>> version when Django receives the data and try to map it to objects via ORM.
>>
>>
>> <https://lh3.googleusercontent.com/-lti9EnCOwpI/WW5My639ZTI/Des/gUJgzX0Wy5IvrGpf2xGDKFbAkoqN7za2gCLcBGAs/s1600/Screenshot%2Bfrom%2B2017-07-18%2B10-59-30.png>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/21e35905-047c-4d57-9f0c-6012d148597c%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-users/21e35905-047c-4d57-9f0c-6012d148597c%40googlegroups.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/4280217c-b3e1-459b-9487-76a2776c323f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Would old version of mysql caused performance issue in Django

2017-07-18 Thread Kevin Yu



Hi All,

I'm rewritting an web application using Django 1.11. When i hooked up to my 
test mysql database (version 5.7), the performance is amazing. The page 
renders within 1 second. However, when I connect to the existing production 
mysql (version 5.1), the page takes more than 10 seconds. 

I installed the debug toolbar, and I found out the the query time is 
actually not the issue. Most of the time are in the CPU. 

I am wondering whether the Django is having issues with older mysql version 
when Django receives the data and try to map it to objects via ORM.



-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/21e35905-047c-4d57-9f0c-6012d148597c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Need help for defining foreign key based on value from another table

2017-06-21 Thread Kevin Yu

I just found out on Django document, there's an example that's for 
manytomany, using 'through' when defining the foreign key. I think this is 
what i need.

Here's the example:

from django.db import models

class Person(models.Model):
name = models.CharField(max_length=128)

def __str__(self):  # __unicode__ on Python 2
return self.name

class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership')

def __str__(self):  # __unicode__ on Python 2
return self.name

class Membership(models.Model):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
invite_reason = models.CharField(max_length=64)




On Wednesday, June 21, 2017 at 2:49:40 PM UTC-7, Kevin Yu wrote:
>
> I'm working with a legacy database so I have to set managed=False in the 
> model. Here's the 3 related tables:
>
>
> class Branches(models.Model):
> name = models.CharField(max_length=128)
> branchpoint_str = models.CharField(max_length=255)
> dev_lead_id = models.IntegerField(blank=True, null=True)
> source = models.CharField(max_length=255)
> state = models.CharField(max_length=255)
> kind = models.CharField(max_length=255)
> desc = models.TextField(blank=True, null=True)
> approved = models.IntegerField()
> for_customer = models.IntegerField()
> deactivated_at = models.DateTimeField(blank=True, null=True)
> created_at = models.DateTimeField(blank=True, null=True)
> updated_at = models.DateTimeField(blank=True, null=True)
> codb_id = models.IntegerField(blank=True, null=True)
> pm_lead_id = models.IntegerField(blank=True, null=True)
> version = models.CharField(max_length=20, blank=True, null=True)
> path_id = models.IntegerField(blank=True, null=True)
> branchpoint_type = models.CharField(max_length=255, blank=True, 
> null=True)
> branchpoint_id = models.IntegerField(blank=True, null=True)
>
> class Meta:
> managed = False
> db_table = 'branches'
> verbose_name_plural = 'Branches'
>
> class Projects(models.Model):
> id = models.AutoField(primary_key=True)
> name = models.CharField(max_length=40, primary_key=True)
> status = models.CharField(max_length=255)
> platform = models.CharField(max_length=255)
> enabled = models.IntegerField()
> path = models.CharField(max_length=128, blank=True, null=True)
> tag_prefix = models.CharField(max_length=64, blank=True, null=True)
> created_at = models.DateTimeField(blank=True, null=True)
> updated_at = models.DateTimeField(blank=True, null=True)
> codb_id = models.IntegerField(blank=True, null=True)
> template = models.CharField(max_length=64, blank=True, null=True)
> image_path = models.CharField(max_length=128, blank=True, null=True)
> repository_id = models.IntegerField(blank=True, null=True)
> number_scheme = models.CharField(max_length=32)
> special_dir = models.CharField(max_length=32, blank=True, null=True)
> project_family_id = models.IntegerField()
> class Meta:
> managed = False
> db_table = 'projects'
> verbose_name_plural = 'projects'
>
> class BranchesProjects(models.Model):
> # project_id = models.IntegerField()
> # branch_id = models.IntegerField()
> project = models.ForeignKey(Projects, on_delete=models.CASCADE)
> branch = models.ForeignKey(Branches, on_delete=models.CASCADE)
>
> class Meta:
> managed = False
> db_table = 'branches_projects'
>
> I have been able to do the join using raw(). However, the return object is 
> rawqueryset. What I want is queryset so that I can use django-filter to 
> process it. My current raw sql is like this:
> Branches.objects.raw(
> '''SELECT br.id, br.name, br.created_at, br.updated_at,
> br.branchpoint_str, br.source
> FROM branches as br
> LEFT JOIN branches_projects as bp
> ON br.id = bp.branch_id 
> WHERE bp.project_id = "%s" AND source != "other"
> ORDER BY updated_at DES'''
>
>  
> My question is, is there a way to achieve the same result using Django's 
> queryset? I've also explored  the idea of using the extra() in django 
> extra 
> <https://docs.djangoproject.com/en/1.11/ref/models/querysets/#django.db.models.query.QuerySet.extra>
>  but 
> it doesn't really work for me.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsu

Need help for defining foreign key based on value from another table

2017-06-21 Thread Kevin Yu

I'm working with a legacy database so I have to set managed=False in the 
model. Here's the 3 related tables:


class Branches(models.Model):
name = models.CharField(max_length=128)
branchpoint_str = models.CharField(max_length=255)
dev_lead_id = models.IntegerField(blank=True, null=True)
source = models.CharField(max_length=255)
state = models.CharField(max_length=255)
kind = models.CharField(max_length=255)
desc = models.TextField(blank=True, null=True)
approved = models.IntegerField()
for_customer = models.IntegerField()
deactivated_at = models.DateTimeField(blank=True, null=True)
created_at = models.DateTimeField(blank=True, null=True)
updated_at = models.DateTimeField(blank=True, null=True)
codb_id = models.IntegerField(blank=True, null=True)
pm_lead_id = models.IntegerField(blank=True, null=True)
version = models.CharField(max_length=20, blank=True, null=True)
path_id = models.IntegerField(blank=True, null=True)
branchpoint_type = models.CharField(max_length=255, blank=True, 
null=True)
branchpoint_id = models.IntegerField(blank=True, null=True)

class Meta:
managed = False
db_table = 'branches'
verbose_name_plural = 'Branches'

class Projects(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=40, primary_key=True)
status = models.CharField(max_length=255)
platform = models.CharField(max_length=255)
enabled = models.IntegerField()
path = models.CharField(max_length=128, blank=True, null=True)
tag_prefix = models.CharField(max_length=64, blank=True, null=True)
created_at = models.DateTimeField(blank=True, null=True)
updated_at = models.DateTimeField(blank=True, null=True)
codb_id = models.IntegerField(blank=True, null=True)
template = models.CharField(max_length=64, blank=True, null=True)
image_path = models.CharField(max_length=128, blank=True, null=True)
repository_id = models.IntegerField(blank=True, null=True)
number_scheme = models.CharField(max_length=32)
special_dir = models.CharField(max_length=32, blank=True, null=True)
project_family_id = models.IntegerField()
class Meta:
managed = False
db_table = 'projects'
verbose_name_plural = 'projects'

class BranchesProjects(models.Model):
# project_id = models.IntegerField()
# branch_id = models.IntegerField()
project = models.ForeignKey(Projects, on_delete=models.CASCADE)
branch = models.ForeignKey(Branches, on_delete=models.CASCADE)

class Meta:
managed = False
db_table = 'branches_projects'

I have been able to do the join using raw(). However, the return object is 
rawqueryset. What I want is queryset so that I can use django-filter to 
process it. My current raw sql is like this:
Branches.objects.raw(
'''SELECT br.id, br.name, br.created_at, br.updated_at,
br.branchpoint_str, br.source
FROM branches as br
LEFT JOIN branches_projects as bp
ON br.id = bp.branch_id 
WHERE bp.project_id = "%s" AND source != "other"
ORDER BY updated_at DES'''

 
My question is, is there a way to achieve the same result using Django's 
queryset? I've also explored  the idea of using the extra() in django extra 

 but 
it doesn't really work for me.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/1581448d-bb69-4e85-a35c-62d0bfad88ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Raw SQL - How to approach paging and lazy fetching

2017-06-21 Thread Kevin Yu

Thanks Jani. I was able to solve my original issue using your suggestion.

On Friday, June 16, 2017 at 12:54:26 PM UTC-7, Jani Tiainen wrote:
>
> Yes, even there isn't real foreign keys in  the database you can still 
> join models.
>
> Have been doing that few times with legacy database. When defining fkey 
> you need to point it to corresponding field in fkey attributes in your 
> model if it isnt your target model pk.
>
> And you can join nonmanged models with managed just fine.
>
> 16.6.2017 19.26 "Kevin Yu" <sams...@gmail.com > kirjoitti:
>
> Hi Jani,
>
> Thanks for the link. I was aware of this link and actually followed the 
> steps when I started the project. I tried using model but then found out 
> when i need to query a join, it doesn't seem to work unless there's a 
> foreign key between the two models, however, given it's a legacy database, 
> i can't touch the schema at all. Is it possible to query join results with 
> managed=False?
>
> Thanks
>
> On Thursday, June 15, 2017 at 11:31:42 PM UTC-7, Jani Tiainen wrote:
>
>> Hi,
>>
>> Even you do have legacy database, you can use unmanaged models and then 
>> leverage full power of ORM [1].
>>
>> Just create models and in their Meta set managed = False and there you 
>> go. 
>>
>> And what comes to paging - you need to somehow pass offset and limit to 
>> your query.
>>
>> [1] https://docs.djangoproject.com/en/1.11/howto/legacy-databases/
>>
>> On 16.06.2017 02:05, Kevin Yu wrote:
>>
>> Hi All, 
>>
>> I am using raw sql to connect to database. The reason we used raw sql 
>> instead of the Django model is because the database is legacy and is being 
>> shared by multiple applications...
>>
>> I have one use case that I'm struggling right now. Basically I have a 
>> page that fetch more than 1000 results. My query is like this:
>>
>> cursor = connection.cursor()
>> cursor.execute('''
>> SELECT br.id, br.name, br.created_at, br.updated_at,
>> br.branchpoint_str, br.source
>> FROM branches as br
>> LEFT JOIN branches_projects as bp
>> ON br.id = bp.branch_id 
>> WHERE bp.project_id = "%s" AND source != "other"
>> ORDER BY updated_at DESC
>>''', [int(project_id)]
>>)
>>
>> Then in my template, I have this:
>> {% for br in special_branches %}
>>
>>   > link="/files/{{build.image_path}}/build{{build.build_number}}/">
>> {{br.name}}
>> {{br.branchpoint}}
>> {{ br.source|upper }}
>> {{br.updated_at}}
>> {{br.built_at}}
>>   
>> {% endfor %}
>>
>> The current problem is this would create a very long page... I am 
>> wondering how to approach this problem so that I can have different page on 
>> the template, and say 100 result per page, when i click the second page, 
>> then  django will fetch result 100-200.
>>
>> Thanks!
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com.
>> To post to this group, send email to django...@googlegroups.com.
>>
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/f6a3df97-8218-4fb8-b200-f4535797e135%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-users/f6a3df97-8218-4fb8-b200-f4535797e135%40googlegroups.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> -- 
>> Jani Tiainen
>>
>> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users...@googlegroups.com .
> To post to this group, send email to django...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/7e4a44fc-e5ce-4562-8b75-09ba065e34f5%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/django-users/7e4a44fc-e5ce-4562-8b75-09ba065e34f5

Re: Raw SQL - How to approach paging and lazy fetching

2017-06-16 Thread Kevin Yu

Hi Jani,

Thanks for the link. I was aware of this link and actually followed the 
steps when I started the project. I tried using model but then found out 
when i need to query a join, it doesn't seem to work unless there's a 
foreign key between the two models, however, given it's a legacy database, 
i can't touch the schema at all. Is it possible to query join results with 
managed=False?

Thanks
On Thursday, June 15, 2017 at 11:31:42 PM UTC-7, Jani Tiainen wrote:
>
> Hi,
>
> Even you do have legacy database, you can use unmanaged models and then 
> leverage full power of ORM [1].
>
> Just create models and in their Meta set managed = False and there you go. 
>
> And what comes to paging - you need to somehow pass offset and limit to 
> your query.
>
> [1] https://docs.djangoproject.com/en/1.11/howto/legacy-databases/
>
> On 16.06.2017 02:05, Kevin Yu wrote:
>
> Hi All, 
>
> I am using raw sql to connect to database. The reason we used raw sql 
> instead of the Django model is because the database is legacy and is being 
> shared by multiple applications...
>
> I have one use case that I'm struggling right now. Basically I have a page 
> that fetch more than 1000 results. My query is like this:
>
> cursor = connection.cursor()
> cursor.execute('''
> SELECT br.id, br.name, br.created_at, br.updated_at,
> br.branchpoint_str, br.source
> FROM branches as br
> LEFT JOIN branches_projects as bp
> ON br.id = bp.branch_id 
> WHERE bp.project_id = "%s" AND source != "other"
> ORDER BY updated_at DESC
>''', [int(project_id)]
>)
>
> Then in my template, I have this:
> {% for br in special_branches %}
>
>link="/files/{{build.image_path}}/build{{build.build_number}}/">
> {{br.name}}
> {{br.branchpoint}}
> {{ br.source|upper }}
> {{br.updated_at}}
> {{br.built_at}}
>   
> {% endfor %}
>
> The current problem is this would create a very long page... I am 
> wondering how to approach this problem so that I can have different page on 
> the template, and say 100 result per page, when i click the second page, 
> then  django will fetch result 100-200.
>
> Thanks!
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users...@googlegroups.com .
> To post to this group, send email to django...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/f6a3df97-8218-4fb8-b200-f4535797e135%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/django-users/f6a3df97-8218-4fb8-b200-f4535797e135%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> -- 
> Jani Tiainen
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/7e4a44fc-e5ce-4562-8b75-09ba065e34f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Raw SQL - How to approach paging and lazy fetching

2017-06-15 Thread Kevin Yu

Hi All,

I am using raw sql to connect to database. The reason we used raw sql 
instead of the Django model is because the database is legacy and is being 
shared by multiple applications...

I have one use case that I'm struggling right now. Basically I have a page 
that fetch more than 1000 results. My query is like this:

cursor = connection.cursor()
cursor.execute('''
SELECT br.id, br.name, br.created_at, br.updated_at,
br.branchpoint_str, br.source
FROM branches as br
LEFT JOIN branches_projects as bp
ON br.id = bp.branch_id 
WHERE bp.project_id = "%s" AND source != "other"
ORDER BY updated_at DESC
   ''', [int(project_id)]
   )

Then in my template, I have this:
{% for br in special_branches %}

  
{{br.name}}
{{br.branchpoint}}
{{ br.source|upper }}
{{br.updated_at}}
{{br.built_at}}
  
{% endfor %}

The current problem is this would create a very long page... I am wondering 
how to approach this problem so that I can have different page on the 
template, and say 100 result per page, when i click the second page, then 
 django will fetch result 100-200.

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/f6a3df97-8218-4fb8-b200-f4535797e135%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Would old version of mysql caused performance issue in Django

Re: Would old version of mysql caused performance issue in Django

Re: Would old version of mysql caused performance issue in Django

Re: Would old version of mysql caused performance issue in Django

Re: Would old version of mysql caused performance issue in Django

Would old version of mysql caused performance issue in Django

Re: Need help for defining foreign key based on value from another table

Need help for defining foreign key based on value from another table

Re: Raw SQL - How to approach paging and lazy fetching

Re: Raw SQL - How to approach paging and lazy fetching

Raw SQL - How to approach paging and lazy fetching

11 matches

Site Navigation

Mail list logo

Footer information