Re: [Tutor] Training, Was: Basic Question about Visualization for enduser
Caveat: This is a long term bug bear of mine and I spent many, many hours over the last 20 years in discussion with our local universities discussing how software-engineering training could be made more relevant to the real world. I'll try to keep it focused :-) On 25/06/2019 21:00, David L Neil wrote: > This illustrates an interesting (at least to me?us) side-effect: in > order to reduce "cognitive load" we often introduce training by using > the most straight-forward or 'simple' tools. In this case Jupyter (which > I think a most marvellous invention). Indeed, and for those with a use for it, extremely valuable and not really very simple! > However, such illustrative devices are often not what is used in the > 'average' development department (assuming there is such a thing). Maybe not, but then Python covers a wide gamut and some researchers for example never get beyond Jupyter, matplotlib etc etc. Those tools deliver all they need. You have to remember that software development is only one sphere in which Python is used. It is also a research tool, a prototyping tool, a personal productivity tool, a devops production scripting tool in addition to an application development language. Many Python users have no need to distribute their software. And for others Python is likely to be just one small component in a much bigger project involving Java, C++, SQL, etc. Professional devs may well need to build distributable apps. And they will use different tools. But there is little point in teaching students about them because there are few standards and different toolsets for just about every combination of app type and OS. In fact just about every web server framework has its own solution. And what you do for Python is different to Java or C++ or Smalltalk or Lisp... And if the project has multiple languages - and most do nowadays then you need a hybrid solution. There is no point in trying to do more than demonstrate some arbitrary example of what it takes to build a distributable package. Then make it clear that this is not a general solution, just one example. > the training meets the knowledge-need (we hope) but not the requirements > of practical application in the work-place. Frankly the biggest problem with most training regimes is that they put way too much focus on coding and programming languages. Professional devs spend much more of their time analysing, designing and testing than they do coding. In my experience very few trainees have any real experience of analysis, at best some rudimentary theoretical knowledge. In the real world the bigger the project the greater the proportion of time spent analyzing to understand exactly what you need to do. Often up to 25% of the total project time-scale. (Of course, those early activities have lower team sizes so the proportion of budget is more like 10%) Fred Brooks made the same point in his classic essay "No Silver Bullet". He talked about the inherent and intrinsic difficulty of software and understanding the essence of the solution as the major challenges. And no toolset (or training course) can shield you from that. At best they can free you to concentrate on it. Design is only covered at a shallow depth - usually restricted to some UML class diagrams and maybe a message sequence chart. Testing is covered at unit test level but rarely at integration or system test levels. And virtually never at UX or performance level. > On the other hand, should training in Python, and more specifically, the > likes of matplotlib, involve any forlorn attempt at universal coverage > of 'the programmer's tool-set'? My personal thought is that the key word there is 'forlorn' You could teach them to be experts in one toolset and in their very first job they discover they have to learn a complete new toolset. The principles will be similar but the practice very different. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Was: Basic Question about Visualization for enduser
On 26/06/19 4:56 AM, Alan Gauld via Tutor wrote: On 25/06/2019 15:52, Sinardy Xing wrote: My question is, how currently all of this great technology glue together and as a final product for the enduser. Because I cant imagine that we install Anaconda Jupyter Notebook at frontend for the enduser to use it, and give end user bunch of *.py You sort of can and there are tools for wrapping it all up in an installable package. More commonly you create a desktop application using the GUI toolkit of your choice and generate the graphical output and display it yourself (using pyplot etc) NB this is NOT intended as a discussion about the specifics of the OP's situation, nor of the OP personally! This illustrates an interesting (at least to me?us) side-effect: in order to reduce "cognitive load" we often introduce training by using the most straight-forward or 'simple' tools. In this case Jupyter (which I think a most marvellous invention). However, such illustrative devices are often not what is used in the 'average' development department (assuming there is such a thing). Thus the training meets the knowledge-need (we hope) but not the requirements of practical application in the work-place. On the other hand, should training in Python, and more specifically, the likes of matplotlib, involve any forlorn attempt at universal coverage of 'the programmer's tool-set'? Your thoughts? -- Regards =dn ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python Imported Code
On 25/06/2019 14:50, stephen.m.sm...@comcast.net wrote:
> using global, but that fails. I also can't seem to use arguments because the
> module that tkinter fires up with this command:
>
> self.process_button = tk.Button(master, text='Process Request', \
> font = ('times', regular_font,
> 'bold'),\
> fg = "blue",
> command=self.Process_Reservation_Request)
>
> does not seem to support the presence of arguments.
There is a common trick to get round that:
self.process_button = tk.Button(master,
text='Process Request',
font = ('times', regular_font,'bold'),
fg = "blue",
command=lambda :self.Process_Reservation_Request(x,y)
)
The lambda defines a function that takes no arguments but
which then calls your function with arguments.
Of course x and y need to be visible within the function
defining the lambda.
> Demo Program - This currently bombs in part 3 - it can't seem to see the
> value/space created in part1. Any help or reference that will clearly show
> Part 1
>
> import nice
> import nice2
> global myGlobal
> myGlobal = "initial value"
global here does nothing. global is only useful inside a function.
Remember that global in Python means module level - it does not
mean visible between modules!
> print(myGlobal)
> nice.changeGlobal()
> print (nice.myGlobal)
> nice2.changeGlobal()
>
> print("nice version = ", nice.myGlobal)
> print("nice2 version = ", nice2.myGlobal)
> print("mainline=", myGlobal)
>
> part 2 - stored as nice.py
>
> def changeGlobal():
>global myGlobal
This tells changeGlobal() that there is a variable called myGlobal
defined at the module level. That is, within nice. It does not
refer to the one you defined in Part 1.
>#print("entering changeGlobal part 1", myGlobal)
>myGlobal = "hello"
This now creates that module level variable with the value 'hello'.
>print("top of myGlobal", myGlobal)
>
>myGlobal="bye"
And this redefines nice.myGlobal to be 'bye'
The final value will now be seen by Part 1 code as
nice.myGlobal with a value of 'bye'.
> Part3 stored as nice2
> myGlobal = "hello"
This creates a new global variable called myGlobal within
the nice2 module. It is visible to Part 1 as nice2.myGlobal
> def changeGlobal():
>global myGlobal
This tells changeGlobal() to use the module level variable
above. ie nice2.myGlobal.
>print("first value = ", nice.myGlobal)
You didn't import nice into nice2 so this should give an error.
>myGlobal="it worked"
>print("in changeGlobal2 =", myGlobal)
This should set the variable nice2.myGlobal to 'it worked'
and that will be seen in Part 1 code as nice2.myGlobal.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Basic Question about Visualization for enduser
On 25/06/2019 15:52, Sinardy Xing wrote: > My question is, how currently all of this great technology glue together > and as a final product for the enduser. Because I cant imagine that we > install Anaconda Jupyter Notebook at frontend for the enduser to use it, > and give end user bunch of *.py You sort of can and there are tools for wrapping it all up in an installable package. More commonly you create a desktop application using the GUI toolkit of your choice and generate the graphical output and display it yourself (using pyplot etc) Or > I read a little bit about Django web framework, I am not sure if my > understanding is correct. Therefore we are using Django as frontend tier > then it connected to backend python server running the calculation and all > other python stuff including the charting and send back the result to > django front end server for end user to consume. More or less although the Django framework may well host the server code too. Django is a web framework so it generates web pages(HTML) as output. Thus if you can generate png files in your python code Django can put links to those files in its HTML output and the users browser will display them. > My question is how is the end to end commonly use by company product, how > they present those charts to end user.? Nowadays its usually via the web. Depending on how fancy you want to get. It could simply be a static graphics file (png/pdf/jpeg etc) or maybe a Flash graphic(not common now) or even a movie generated by back end code. Or it could be something more dynamic controlled by Javascript in the browser in which case your backend code simply serves up raw data and the Javascript code turns it into dynamic graphics within the browser screen, possibly via SVG or similar technology. There have never been so many options for delivering content to end users! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] tree or link-list questions.
On 25/06/2019 13:24, mhysnm1...@gmail.com wrote:
... the tree I am trying to create is:
>
> * A single root node
> * A node can only have a single parent.
> * A parent can have multiple children.
> * Each node is unique.
A tree with multiple children (where multiple is more than 2)
is entirely possible but does make the code more complex.
Have you considered a binary tree where items go either left or right
depending on whether they are "greater than" or "less than" (for some
arbitrary definition of greater and less) than the parent node?
This is much simpler to code, and you will find lots of examples
online.
-- monospace font needed ---
Root
the
brown
fox
cow
car
yellow
car
green
house
quick
brown
fox
yellow
flowers
-- end of monospace font ---
The immediate questions you must answer (because only
you understand the requirements) are:
1) How do you know which tree branch to navigate down?
For example you have two car nodes. If searching for car
which is the correct answer? Similarly for fox?
2) How do you know to go down an extra level. For example
there are 4 nodes below 'the' What determines whether a
new node sits alongside the existing nodes or at a level
below.
3) Having decided you need to go a level lower how do
you know which subnode to go down? How do you insert the
node car into the correct subnode of the?(either brown
or yellow)
It looks like you are doing this based on the textual context.
That means you insertion code has to be stateful since
it must remember the previous word inserted (or have access
to the original data). That gets complicated really quickly.
Similarly, when you try to retrieve the data how do yu know which node
to fetch. For example if I search for fox? Surely you will need to
provide the full 'path' to fox topick the right one? But if you know the
path why search a tree?
In that case nested dictionaries would seem a better solution?
Something like(missing all the quotes!):
data = {
the: [
{brown: [
{fox:[]},
{cow:[]},
{car:[]},
]},
{yellow: [
{car: []}
]},
{green: [
{house: []}
]},
{quick: [
{brown: [
{fox: []}
]},
]},
]},
{yellow: [
{flowers: []}
]}
]}
Then you can access the appropriate path:
myFox = data[the][quick][brown][fox]
Just a thought. Since I don't really know how you intend
to use this I'm not sure if that would work for you.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] Python Imported Code
Introduction:
I have written a 'program' that does some reasonable screen scraping off of
a specific website. The program has gotten too large so I have tried to
segment it into logical pieces (tkinter logic as a start) but I am having
problems. Specifically I need to pass several dictionaries to the module
(imported code) that validates some user selection and into the code that
navigates through the website. I have also created a variable that is set to
0 when a valid entry has been made by the user (telling the scraper what to
do) that needs to be checked in the scraper module that is waiting patiently
before it starts. There are multiple threads working as well because I may
need to run several session as once.
After struggling with my basic need - the ability to pass
variables/dictionaries across modules, especially imported modules, I have
read everything I can find and tried small, demo programs. But is till can't
get it
What I need:
An ability to create dictionaries (that have validation information in them)
and variables (that signal status and contain information entered by the
user in tinkter) that I can make available to all components. I have tried
using global, but that fails. I also can't seem to use arguments because the
module that tkinter fires up with this command:
self.process_button = tk.Button(master, text='Process Request', \
font = ('times', regular_font,
'bold'),\
fg = "blue",
command=self.Process_Reservation_Request)
does not seem to support the presence of arguments.
Demo Program - This currently bombs in part 3 - it can't seem to see the
value/space created in part1. Any help or reference that will clearly show
me how to solve this problem would be most appreciated. I have read a number
of points that recommend against using global anywhere (it does not seem to
work with imported code) but I just can't find a recommendation for doing
something that seems pretty fundamental.
Thanks to an and all that try to help!
Part 1
import nice
import nice2
global myGlobal
myGlobal = "initial value"
print(myGlobal)
nice.changeGlobal()
print (nice.myGlobal)
nice2.changeGlobal()
print("nice version = ", nice.myGlobal)
print("nice2 version = ", nice2.myGlobal)
print("mainline=", myGlobal)
part 2 - stored as nice.py
def changeGlobal():
global myGlobal
#print("entering changeGlobal part 1", myGlobal)
myGlobal = "hello"
print("top of myGlobal", myGlobal)
myGlobal="bye"
print("changeGlobal =", myGlobal)
#changeGlobal()
Part3 stored as nice2
myGlobal = "hello"
def changeGlobal():
global myGlobal
print("first value = ", nice.myGlobal)
myGlobal="it worked"
print("in changeGlobal2 =", myGlobal)
#changeGlobal()
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Basic Question about Visualization for enduser
Hi Mats,
Thanks for the reply. I am so sorry the question was not clear. I was
rushing home when composing the question.
I have learned python and a little about the pandas lib also the
visualization lib such as maplotlib, I also learn on how to generate the
output (thanks again for that example I learned that too few days ago).
My question is, how currently all of this great technology glue together
and as a final product for the enduser. Because I cant imagine that we
install Anaconda Jupyter Notebook at frontend for the enduser to use it,
and give end user bunch of *.py
I read a little bit about Django web framework, I am not sure if my
understanding is correct. Therefore we are using Django as frontend tier
then it connected to backend python server running the calculation and all
other python stuff including the charting and send back the result to
django front end server for end user to consume.
My question is how is the end to end commonly use by company product, how
they present those charts to end user.?
Thanks very much for your kind reply, I am very new in this and all the
information is via selfstudy and online document including this forum.
Regards,
Sinardy
On Tue, Jun 25, 2019 at 10:27 PM Mats Wichmann wrote:
> On 6/25/19 6:39 AM, Sinardy Xing wrote:
> > Hi,
> >
> > I am a newbie with python and the data visualization.
> > I have completed pandas.DataFrame stuff and also the matplotlib.
> >
> > All of the example that I learned from internet currently are using the
> > Anaconda Jupyter Notebook.
> > I know there are API where we can output the result of the graph to png,
> > will that be possible all these done automatically and dynamically via an
> > apps ?
>
> You can generate graphs with mathplotlib. If you're specifically
> looking for a png, it looks a bit like this (assuming you want to follow
> the usual conventions for shortening the name, which certainly isn't
> required):
>
> from matplotlib import pyplot as plt
>
> plt.savefig('foo.png')
>
> savefig looks at the file extension in determining what to output.
>
> Without a specific question, I'm not sure what else we can say...
>
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] tree or link-list questions.
On 25/06/2019 13:24, mhysnm1...@gmail.com wrote: A much less open ended question :-) > class Node: > > def __init__(self, name, value): > self.parent = None > self.child = [] This should be plural if its a list. So call it children... However I'm not really sure it should be a list, most trees have two child nodes - called left and right by convention. But some trees do have multiple children, it's up to you. > self.name = name > self.value = value > > def add_child(self, name, value): > # Compare the new value with the parent node > if self.name: > if name != self.name: > if self.child is None: > self.parent = self.child > self.child = Node(name, value) > else: > self.child.add_child(name, value) > else: > self.name = name > self.value = value > My first comment would be that in OOP we should be trying to use objects so rather than pass name,value into the method pass another Node instance: def add_child(self, newNode): if newNode.name != self.name self.children.append(newNode) newNode.parent = self And if necessary call it like myNode.add_child(Node(name, value)) However, looking at your code above you seem to be confused about which node it adding which. self is the top level node to which you are adding a child. So when finished you want the existing nodes parent to be untouched and it's children list to include the new node. Instead you are overwriting the list with the new Node instance and setting the top level node's parent to the child. You really want the child to have the top level as its parent. The initial test of if self.name shouldn't be necessary since you assign name when you create the node. Without a name the node is pointless so you should check if name is None in the init method and throw an exception (ValueError?) if name is not valid. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Basic Question about Visualization for enduser
On 25/06/2019 13:39, Sinardy Xing wrote: > All of the example that I learned from internet currently are using the > Anaconda Jupyter Notebook. > I know there are API where we can output the result of the graph to png, > will that be possible all these done automatically and dynamically via an > apps ? If you mean "is it possible to write an app that does it all?" then the answer is yes. If you mean "is there an existing app that does it all?" the answer is maybe. It depends on exactly what you want to do, what kind of data you have etc. But a Google search is probably your best bet. Python itself is just a programming language so somebody - possibly you! - has to write the apps. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Basic Question about Visualization for enduser
On 6/25/19 6:39 AM, Sinardy Xing wrote:
> Hi,
>
> I am a newbie with python and the data visualization.
> I have completed pandas.DataFrame stuff and also the matplotlib.
>
> All of the example that I learned from internet currently are using the
> Anaconda Jupyter Notebook.
> I know there are API where we can output the result of the graph to png,
> will that be possible all these done automatically and dynamically via an
> apps ?
You can generate graphs with mathplotlib. If you're specifically
looking for a png, it looks a bit like this (assuming you want to follow
the usual conventions for shortening the name, which certainly isn't
required):
from matplotlib import pyplot as plt
plt.savefig('foo.png')
savefig looks at the file extension in determining what to output.
Without a specific question, I'm not sure what else we can say...
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python
On Jun 25, 2019 8:52 AM, "Shaon Debnath" wrote: > > I just wanted to know all about map() function in python. See https://www.geeksforgeeks.org/python-map-function/. If after reading that you still have questions please come back and ask them. Bob Gailer ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] python
I just wanted to know all about map() function in python. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Basic Question about Visualization for enduser
Hi, I am a newbie with python and the data visualization. I have completed pandas.DataFrame stuff and also the matplotlib. All of the example that I learned from internet currently are using the Anaconda Jupyter Notebook. I know there are API where we can output the result of the graph to png, will that be possible all these done automatically and dynamically via an apps ? Thanks Kind regards Sinardy ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] tree or link-list questions.
All,
Windows 10, python 3.6.
I am trying to create a data structure. I might be using the wrong term here,
but I am calling it a hierarchical tree. The properties of the tree I am trying
to create is:
* A single root node
* A node can only have a single parent.
* A parent can have multiple children.
* Each node is unique.
This is also my first attempt in doing Object coding ever. Thus the terminology
I am not going to use correct.
Data structure:
records = [‘the brown fox',
‘the brown cow’,
‘the brown car’,
‘the yellow car’,
‘the green house’,
‘yellow flowers’,
‘the quick brown fox’]
What I am trying to do:
I want to create a tree structure that stores the first word under the root,
the 2nd word under the first word as a child, ETC. the word ‘the’ for example
can only occur once as a node. Below is an example of the tree structure I am
trying to achieve:
Root
Root / the
Root / the / brown
Root / the / brown / fox
Root / the / brown / cow
Root / the / brown / car
Root / the / yellow
Root / the / yellow / car
Root / the / green
Root / the / green / house
Root / the / quick
Root / the / quick / brown
Root / the / quick / brown / fox
Root / yellow
Root / yellow / flowers
The code I have used to break up the words and build a dictionary to identify
the unique words plus the hit count:
nar_dict = {}
for text in records:
words = text.split()
l = len(words)
for i in range(1, l+1):
# only method I have found to correctly join the words.
tmp_list = words[:i]
key = ' '.join(tmp_list)
nar_dict[key] = 0
# perform the searches using dictionary key and count the number of hits.
for key, item in nar_dict.items():
print (key, item)
for line in records:
if line.startswith(key):
nar_dict[key] += 1
The result is (note, the output below is using a different data set):
{'the': 5, 'the brown': 3, 'the brown cow': 2, 'the brown car': 1, 'the
yellow': 1, 'the yellow car': 1, 'yellow': 2, 'yellow house': 1, 'the quick':
1, 'the quick fox': 1, 'yellow flowers': 1}
The code below works fine if I want to have a single parent and a single child.
The init definition I have used an empty array to store the children. I am not
sure how I can reference the new node back to the parent node. I have looked at
some packages like anytree and I got completely lost with the code. I have
looked for examples on the net and did not find anything I could understand.
Here is my code thus far with the children as a list.
# Value = hit counter
# Name = node name
class Node:
def __init__(self, name, value):
self.parent = None
self.child = []
self.name = name
self.value = value
def add_child(self, name, value):
# Compare the new value with the parent node
if self.name:
if name != self.name:
if self.child is None:
self.parent = self.child
self.child = Node(name, value)
else:
self.child.add_child(name, value)
else:
self.name = name
self.value = value
# Print the tree
def PrintTree(self):
if self.child:
self.child.PrintTree()
print( self.name, self.value),
I want to be able to do the following syntax:
Node.children[-1]value
Child = Node.children[0].children[0]
Parent = child.parent.name
I know I will have to write a search and walk functionality.
Any comments and tips on how to fix the parent issue. I would be grateful.
Hopefully this is not to open ended. Done my best to be very specific.
Sean – Open ended question master
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
On 24/06/2019 17:15, johnf wrote: > def locChoices(self): > locDS = self.eslocation.getDataSet() > loc_Choices=[''] > locKeys=[0] > for row in locDS: > loc_Choices.append(row['facility']) > locKeys.append(row['pkid']) > > return loc_Choices,locKeys > ... and wonder if it will improve the performance. Because you are building two lists in one loop it probably won't do much for performance. It might even be worse. As always measure. But something else that might make a small difference is the pre-fetch of the locations. Why not: for location in self.eslocation.getDataSet(): You don't use the locDS value anywhere other than in the loop. Also the choice of method name is unfortunate in that you return both choices and keys but the method name implies only choices. Minor nit-picks... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
