Re: Error pd.set_option('display.width', 10000)

[MRAB via Python-list]

On 2024-02-03 23:02, gelukt gelukt via Python-list wrote:

Dear,

While running a code, I get the error below:
What does this error mean? How can I fix this error?

C:\Users\brech\Desktop\Crypto\venv\Scripts\python.exe 
"C:/Users/brech/Desktop/Crypto/Project/aaa Arbitrage.py"
Traceback (most recent call last):
   File "C:\Users\brech\Desktop\Crypto\Project\aaa Arbitrage.py", line 458, in 

 methode()
   File "C:\Users\brech\Desktop\Crypto\Project\aaa Arbitrage.py", line 49, in 
methode
 wb.sheets[website]['A' + str(1+a)].value = 
inputString[startIndex:startIndex + 3]
 
   File "C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\main.py", 
line 2411, in value
 conversion.write(data, self, self._options)
   File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\__init__.py",
 line 102, in write
 pipeline(ctx)
   File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\framework.py",
 line 79, in __call__
 stage(*args, **kwargs)
   File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\standard.py",
 line 75, in __call__
 self._write_value(ctx.range, ctx.value, scalar)
   File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\standard.py",
 line 63, in _write_value
 rng.raw_value = value
 ^
   File "C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\main.py", 
line 1973, in raw_value
 self.impl.raw_value = data
 ^^^
   File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\_xlwindows.py", 
line 1209, in raw_value
 self.xl.Value = data
 ^
   File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\_xlwindows.py", 
line 161, in __setattr__
 return setattr(self._inner, key, value)

   File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\win32com\client\__init__.py",
 line 597, in __setattr__
 self._oleobj_.Invoke(*(args + (value,) + defArgs))
pywintypes.com_error: (-2147352567, 'Er is een uitzondering opgetreden.', (0, 
None, None, None, 0, -2147024882), None)

Process finished with exit code 1


There's a question on StackOverflow that looks similar:

https://stackoverflow.com/questions/72223654/pywintypes-com-error-2147352567-exception-occured-0-none-none-none-0

Something to do with the cell format.

Check which cell it i, what value it's trying to put there, and whether 
the format of the cell and the value are compatible.

--
https://mail.python.org/mailman/listinfo/python-list

Error pd.set_option('display.width', 10000)

[gelukt gelukt via Python-list]

Dear,

While running a code, I get the error below:
What does this error mean? How can I fix this error?

C:\Users\brech\Desktop\Crypto\venv\Scripts\python.exe 
"C:/Users/brech/Desktop/Crypto/Project/aaa Arbitrage.py"
Traceback (most recent call last):
  File "C:\Users\brech\Desktop\Crypto\Project\aaa Arbitrage.py", line 458, in 

methode()
  File "C:\Users\brech\Desktop\Crypto\Project\aaa Arbitrage.py", line 49, in 
methode
wb.sheets[website]['A' + str(1+a)].value = 
inputString[startIndex:startIndex + 3]

  File "C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\main.py", 
line 2411, in value
conversion.write(data, self, self._options)
  File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\__init__.py",
 line 102, in write
pipeline(ctx)
  File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\framework.py",
 line 79, in __call__
stage(*args, **kwargs)
  File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\standard.py",
 line 75, in __call__
self._write_value(ctx.range, ctx.value, scalar)
  File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\conversion\standard.py",
 line 63, in _write_value
rng.raw_value = value
^
  File "C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\main.py", 
line 1973, in raw_value
self.impl.raw_value = data
^^^
  File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\_xlwindows.py", 
line 1209, in raw_value
self.xl.Value = data
^
  File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\xlwings\_xlwindows.py", 
line 161, in __setattr__
return setattr(self._inner, key, value)
   
  File 
"C:\Users\brech\Desktop\Crypto\venv\Lib\site-packages\win32com\client\__init__.py",
 line 597, in __setattr__
self._oleobj_.Invoke(*(args + (value,) + defArgs))
pywintypes.com_error: (-2147352567, 'Er is een uitzondering opgetreden.', (0, 
None, None, None, 0, -2147024882), None)

Process finished with exit code 1

Kind regards

Gelukt
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Await expressions (Posting On Python-List Prohibited)

[Mild Shock via Python-list]

Funny source code tells me IOCP is used;

proactor is only implemented on Windows with IOCP.
https://github.com/python/cpython/blob/3.12/Lib/asyncio/proactor_events.py

But maybe the focus is more on networking than file system.

But it has sock_sendfile() that might avoid copying data to userspace.

Mild Shock schrieb:

The docu tells me:

Windows
loop.add_reader() and loop.add_writer() only accept
socket handles (e.g. pipe file descriptors are not supported).
https://docs.python.org/3/library/asyncio-platforms.html

Alternatives are aiofiles and anyio and maybe more,
but not sure whether they span all platforms, i.e. unix,
windows and mac or whether they have similar restrictions.

Theoretically on Windows IOCP should also cover file handles:
https://en.wikipedia.org/wiki/Input/output_completion_port

Its use by libuv the basis for node.js.

Lawrence D'Oliveiro schrieb:

On Sat, 3 Feb 2024 00:45:18 +0100, Mild Shock wrote:


... that works on windows ...


You lost me there.





--
https://mail.python.org/mailman/listinfo/python-list

Re: Await expressions (Posting On Python-List Prohibited)

[Mild Shock via Python-list]

The docu tells me:

Windows
loop.add_reader() and loop.add_writer() only accept
socket handles (e.g. pipe file descriptors are not supported).
https://docs.python.org/3/library/asyncio-platforms.html

Alternatives are aiofiles and anyio and maybe more,
but not sure whether they span all platforms, i.e. unix,
windows and mac or whether they have similar restrictions.

Theoretically on Windows IOCP should also cover file handles:
https://en.wikipedia.org/wiki/Input/output_completion_port

Its use by libuv the basis for node.js.

Lawrence D'Oliveiro schrieb:

On Sat, 3 Feb 2024 00:45:18 +0100, Mild Shock wrote:


... that works on windows ...


You lost me there.



--
https://mail.python.org/mailman/listinfo/python-list

Re: Await expressions (Posting On Python-List Prohibited)

[Mild Shock via Python-list]

And whats the roadmap for an asyncified module
loader, is this on the radar of Python?

Mild Shock schrieb:


I am still waiting for async files in the
style of nodejs that works on windows and

is bundled with the main python distribution.
I am not very  fond on doing something

like adding listeners to a file descriptor,
in nodejs async files are based on callbacks

not on listeners. Whats the roadmap here?

Lawrence D'Oliveiro schrieb:

On 2 Feb 2024 09:12:06 GMT, Stefan Ram wrote:


|The IO part of the event loop is built upon a single crucial
|function called "select".


select(2) has limitations. Better to use poll(2). Depending on *nix
variant, other more advanced alternatives may also be available
.





--
https://mail.python.org/mailman/listinfo/python-list

Re: Await expressions (Posting On Python-List Prohibited)

[Mild Shock via Python-list]

I am still waiting for async files in the
style of nodejs that works on windows and

is bundled with the main python distribution.
I am not very  fond on doing something

like adding listeners to a file descriptor,
in nodejs async files are based on callbacks

not on listeners. Whats the roadmap here?

Lawrence D'Oliveiro schrieb:

On 2 Feb 2024 09:12:06 GMT, Stefan Ram wrote:


|The IO part of the event loop is built upon a single crucial
|function called "select".


select(2) has limitations. Better to use poll(2). Depending on *nix
variant, other more advanced alternatives may also be available
.



--
https://mail.python.org/mailman/listinfo/python-list

RE: Extract lines from file, add to new files

[AVI GROSS via Python-list]

Dave,

You and I have had some experience in teaching or tutoring others and I think 
it fair to say our motivation is closer to teaching someone how they can fish 
for themselves rather than just handing them a fully-cooked fish.

My favorite kinds of questions, thus, include someone who explains what they 
are trying to do and shows some code along with indications of what it produced 
(including error messages) and what it should produce. Then the question should 
not be a request to just FIX THIS or WRITE IT FOR ME but asking if someone can 
show what they did wrong with some idea where it went wrong.

This may not be so common but it allows faster and easier help.

But others out there probably can be helped including people who want 
suggestions of approaches to try.

We have discussed some issues here and elsewhere where some purists tend to 
want to stick with close to basic aspects of the language. In some R 
communities, some get upset if someone asks a question (or others supply an 
answer) using non-official add-ons called the tidyverse and I suspect some here 
don't really like discussions that focus mainly on numpy/pandas or so many 
other modules. Yet, the reality is, that except for students who are supposed 
to be learning the basics of the language and show solutions compatible with 
what is being taught in class, others can and should be open to simple 
solutions using non-standard tools that are widely available and vetted. If a 
student is asked to use some method to calculate a value like pi, then 
suggesting they use math.pi is not useful. But if the assignment is to do 
something in trigonometry, it seems a good thing to use as a part of a larger 
solution rather than either embedding a constant to N decimal places or 
calculating it on the f
 ly.

I think people like us who volunteer to reply should consider our choices too. 
I think it is fair to reply, as I saw on the tutor forum, that the code shown 
uses a method that is not the way the replier would do it but nonetheless offer 
some thoughts on particular python coding errors. I am talking about someone 
who wanted to implement a binary tree using a numpy array or something with 
minimal explanation and demanded a fix. They never supplied enough info and I 
think the few who replied backed away. As you mention, they seemed resistant to 
starting over and using data structures that are perhaps more amenable, and in 
any case, the code shown seemed mostly focused on how to make random and 
non-redundant names. 

I will end with a comment. I have heard of interview techniques for a job where 
they deliberately supply a problem in which the goal is not so much to be easy 
to solve in front of them in real time but to watch how the person looking for 
a job responds to the uncertainties and asks follow-up questions or verbalizes 
things like, if it is like this, I might use this technique but if you also 
need that then ...

So, I shudder to think what happens if someone being interviewed turns around 
and asks us and further confuses things with changes to make it harder to 
recognize they are asking for outside help. The answer expected may well be to 
NOT use say the older versions of PASCAL to do something but switch to 
something better suited (and for that matter available.)  I would not want to 
program the DES encryption/decryption method in Pascal again! And these days, 
it seems much better to just find a module or package that meets such needs.

Avi



-Original Message-
From: Python-list  On 
Behalf Of dn via Python-list
Sent: Saturday, February 3, 2024 5:02 PM
To: python-list@python.org
Subject: Re: Extract lines from file, add to new files

Every trainer, in any field, has to deal with these problems - all the 
time, and over-and-over.


On 4/02/24 06:58, Thomas Passin via Python-list wrote:
> In my view this whole thread became murky and complicated because the OP 
> did not write down the requirements for the program.  Requirements are 
> needed to communicate with other people.  An individual may not need to 
> actually write down the requirements - depending on their complexity - 
> but they always exist even if only vaguely in a person's mind.  The 
> requirements may include what tools or languages the person wants to use 
> and why.
> 
> If you are asking for help, you need to communicate the requirements to 
> the people you are asking for help from.
> 
> The OP may have thought the original post(s) contained enough of the 
> requirements but as we know by now, they didn't.

There is another possible interpretation in such situations (not 
necessarily this one): that the person is fixated on a particular 
solution (and unable/unwilling to adjust his/her thinking to consider 
more widely).

Thus, the question is not: 'here's an entire problem, how can it be 
solved', but more: 'I have a solution, and want help to implement it 
(and only it) just-so'.


The latter is an interesting psychology:

1
an 

Re: Extract lines from file, add to new files

[Thomas Passin via Python-list]

On 2/3/2024 5:02 PM, dn via Python-list wrote:
Every trainer, in any field, has to deal with these problems - all the 
time, and over-and-over.



On 4/02/24 06:58, Thomas Passin via Python-list wrote:
In my view this whole thread became murky and complicated because the 
OP did not write down the requirements for the program.  Requirements 
are needed to communicate with other people.  An individual may not 
need to actually write down the requirements - depending on their 
complexity - but they always exist even if only vaguely in a person's 
mind.  The requirements may include what tools or languages the person 
wants to use and why.


If you are asking for help, you need to communicate the requirements 
to the people you are asking for help from.


The OP may have thought the original post(s) contained enough of the 
requirements but as we know by now, they didn't.


There is another possible interpretation in such situations (not 
necessarily this one): that the person is fixated on a particular 
solution (and unable/unwilling to adjust his/her thinking to consider 
more widely).


Thus, the question is not: 'here's an entire problem, how can it be 
solved', but more: 'I have a solution, and want help to implement it 
(and only it) just-so'.



The latter is an interesting psychology:

1
an experienced person who is trying to translate from one tool to 
another (Python), but discovers that a word-for-word solution is 
difficult because of the artificial-constraints they've placed on the 
situation.


2
a beginner who doesn't know what (s)he doesn't know and comes-up with an 
idea, but fails to appreciate that there is likely more than one path to 
the goal.



The person asking for help may not realize they don't know enough to 
write down all the requirements; an effort to do so may bring that 
lack to visibility.


In the case of 'Beginners' this should probably be taken-as-read!

Which is why we will always find ourselves asking questions or 'please 
give more information'...



However, there are other reasons, eg corporate concerns or personality; 
why people don't want to give more information. The former is reasonable 
(have suffered from same myself). The latter may reveal that the person 
is 'difficult to deal with'...



Mailing lists like these have a drawback that it's hard to impossible 
for someone not involved in a thread to learn anything general from 
it. We can write over and over again to please state clearly what you 
want to do and where the sticking points are, but newcomers post new 
questions without ever reading these pleas.  Then good-hearted people 
who want to be helpful end up spending a lot of time trying to guess 
what is actually being asked for, and maybe never find out with enough 
clarity.  Others take a guess and then spend time working up a 
solution that may or may not be on target.


So please! before posting a request for help, write down the 
requirements as best you can figure them out, and then make sure that 
they are expressed such that the readers can understand.


Unfortunately, if the person doesn't understand the problem (leave-aside 
any ideas of solution), then (s)he will not be able to clearly 
communicate same to us, in any way, shape, or form...


Which brings one to the question: if a person cannot express the problem 
clearly and completely, is (s)he suited to development work? If the 
problem is not understood, could 'the solution' ever be more than an 
exercise in hope?

(prototyping and experimentation aside)


Pairs programming can be fun and productive, if you are lucky to have 
the right person to work with.  I've had one person like that over the 
years.


Yes, it is frustrating to invest time and effort in helping someone, 
only for same to disappear 'into a black hole'. The lack of response 
seems to indicate a lack of respect or appreciation. Is this perhaps 
part of today's "consumer" life-style, where so few are contributors or 
creators?



On the other side of that coin: do the people who make assumptions and 
(kindly) blaze-ahead with 'a solution', actually help the conversation? 
If the assumptions are correct, yes! What if they are not?



...and don't get me started on folk who want us to do their 
training-assignments or build some application, for them!



As a slight aside: on one training-course DiscussionList/BulletinBoard 
set-up, if a trainee asked a question without a descriptive 
title/SubjectLine, eg "Python not working" or "Urgent: please help"; I 
asked them to re-post with a title that would help others in a similar 
situation find the topic - and closed the original thread.


Some found it "brutal" - probably skewing towards those who felt 
"Urgent" because they'd left things too close to deadline. Others joined 
the (later) thread because they could identify the topic and realise 
their interest in learning or contributing to the conversation...



Time pressures lead to a multitude of evils!


There's a 

Re: Extract lines from file, add to new files

[dn via Python-list]

Every trainer, in any field, has to deal with these problems - all the 
time, and over-and-over.



On 4/02/24 06:58, Thomas Passin via Python-list wrote:
In my view this whole thread became murky and complicated because the OP 
did not write down the requirements for the program.  Requirements are 
needed to communicate with other people.  An individual may not need to 
actually write down the requirements - depending on their complexity - 
but they always exist even if only vaguely in a person's mind.  The 
requirements may include what tools or languages the person wants to use 
and why.


If you are asking for help, you need to communicate the requirements to 
the people you are asking for help from.


The OP may have thought the original post(s) contained enough of the 
requirements but as we know by now, they didn't.


There is another possible interpretation in such situations (not 
necessarily this one): that the person is fixated on a particular 
solution (and unable/unwilling to adjust his/her thinking to consider 
more widely).


Thus, the question is not: 'here's an entire problem, how can it be 
solved', but more: 'I have a solution, and want help to implement it 
(and only it) just-so'.



The latter is an interesting psychology:

1
an experienced person who is trying to translate from one tool to 
another (Python), but discovers that a word-for-word solution is 
difficult because of the artificial-constraints they've placed on the 
situation.


2
a beginner who doesn't know what (s)he doesn't know and comes-up with an 
idea, but fails to appreciate that there is likely more than one path to 
the goal.



The person asking for help may not realize they don't know enough to 
write down all the requirements; an effort to do so may bring that lack 
to visibility.


In the case of 'Beginners' this should probably be taken-as-read!

Which is why we will always find ourselves asking questions or 'please 
give more information'...



However, there are other reasons, eg corporate concerns or personality; 
why people don't want to give more information. The former is reasonable 
(have suffered from same myself). The latter may reveal that the person 
is 'difficult to deal with'...



Mailing lists like these have a drawback that it's hard to impossible 
for someone not involved in a thread to learn anything general from it. 
We can write over and over again to please state clearly what you want 
to do and where the sticking points are, but newcomers post new 
questions without ever reading these pleas.  Then good-hearted people 
who want to be helpful end up spending a lot of time trying to guess 
what is actually being asked for, and maybe never find out with enough 
clarity.  Others take a guess and then spend time working up a solution 
that may or may not be on target.


So please! before posting a request for help, write down the 
requirements as best you can figure them out, and then make sure that 
they are expressed such that the readers can understand.


Unfortunately, if the person doesn't understand the problem (leave-aside 
any ideas of solution), then (s)he will not be able to clearly 
communicate same to us, in any way, shape, or form...


Which brings one to the question: if a person cannot express the problem 
clearly and completely, is (s)he suited to development work? If the 
problem is not understood, could 'the solution' ever be more than an 
exercise in hope?

(prototyping and experimentation aside)


Yes, it is frustrating to invest time and effort in helping someone, 
only for same to disappear 'into a black hole'. The lack of response 
seems to indicate a lack of respect or appreciation. Is this perhaps 
part of today's "consumer" life-style, where so few are contributors or 
creators?



On the other side of that coin: do the people who make assumptions and 
(kindly) blaze-ahead with 'a solution', actually help the conversation? 
If the assumptions are correct, yes! What if they are not?



...and don't get me started on folk who want us to do their 
training-assignments or build some application, for them!



As a slight aside: on one training-course DiscussionList/BulletinBoard 
set-up, if a trainee asked a question without a descriptive 
title/SubjectLine, eg "Python not working" or "Urgent: please help"; I 
asked them to re-post with a title that would help others in a similar 
situation find the topic - and closed the original thread.


Some found it "brutal" - probably skewing towards those who felt 
"Urgent" because they'd left things too close to deadline. Others joined 
the (later) thread because they could identify the topic and realise 
their interest in learning or contributing to the conversation...



Time pressures lead to a multitude of evils!


There's a quotation that goes something like "the poor will be with your 
always"?

(?possibly Biblical)

Whether we (here) are talking about 'poor' manners, 'poor' 
understanding, 'poor' communication skills, 'poor' 

RE: Extract lines from file, add to new files

[AVI GROSS via Python-list]

We substantially agree with that, Thomas.  In the best of all possible
worlds, someone who gets stuck will sit down and try to carefully spell out
things in ways like you mention and, incidentally, may often catch the error
or figure out how to do it and not even send in a request! LOL!

I think a main thing that the OP and others can do is to not just be
abstract but supply a small example including what output they expect and
perhaps what they did not receive properly along with error messages.

Rich had not tried doing what he wanted in python, yet. I don't know if he
did any other parts yet. I think he was still in a somewhat abstract design
state and hoping for someone to push him at something like a resource to
continue and he did accept suggestions on what now seem somewhat random
things to read as people guessed.

In a sense, I and others can take some blame for the way we widened the
problem while trying to look at it our own way.

But looking back, I think Rich asked close to what he wanted. An example
might have helped such as:

'I have information about multiple clients including an email address and a
human name such as:

u...@domain.com Sally
f...@bark.com Barky

I want to save the info in a file or maybe two in such a way that when I
write a program and ask it to send an email to "f...@bark.com" then it finds
an entry that matches the email address and then uses that to find the
matching name and sends a specified message with a specific salutation in
front like "Dear Barky,".

I was thinking of having two files with one having email address after
another and the other having the corresponding names. I want to use python
to search one file and then return the name in the other. Are there other
and perhaps better ways commonly used to associate one keyword with a
value?'

Back to me. The above is not polished but might still have engendered a
discussion such as how to keep the files synchronized. Yes, the OP could
have added endless clauses saying that they are not asking how to create the
data files or keep them synchronized but just how to match them. The reply
in this case possibly could have suggested they count the lines they have
read until the match and, assuming no blank lines are used, read a second
file till the nth line. We might have been done quickly and THEN had a long
discussion about other ways!

I have participated, like you, in another forum designed for tutoring and I
think the rules and expectations there may be a bit different. Over here, it
is fairer to expect people to take a bit of time and ask clearer questions. 

We live and we learn and then Alzheimer's ...



-Original Message-
From: Python-list  On
Behalf Of Thomas Passin via Python-list
Sent: Saturday, February 3, 2024 12:59 PM
To: python-list@python.org
Subject: Re: Extract lines from file, add to new files

In my view this whole thread became murky and complicated because the OP 
did not write down the requirements for the program.  Requirements are 
needed to communicate with other people.  An individual may not need to 
actually write down the requirements - depending on their complexity - 
but they always exist even if only vaguely in a person's mind.  The 
requirements may include what tools or languages the person wants to use 
and why.

If you are asking for help, you need to communicate the requirements to 
the people you are asking for help from.

The OP may have thought the original post(s) contained enough of the 
requirements but as we know by now, they didn't.

The person asking for help may not realize they don't know enough to 
write down all the requirements; an effort to do so may bring that lack 
to visibility.

Mailing lists like these have a drawback that it's hard to impossible 
for someone not involved in a thread to learn anything general from it. 
We can write over and over again to please state clearly what you want 
to do and where the sticking points are, but newcomers post new 
questions without ever reading these pleas.  Then good-hearted people 
who want to be helpful end up spending a lot of time trying to guess 
what is actually being asked for, and maybe never find out with enough 
clarity.  Others take a guess and then spend time working up a solution 
that may or may not be on target.

So please! before posting a request for help, write down the 
requirements as best you can figure them out, and then make sure that 
they are expressed such that the readers can understand.

On 2/3/2024 11:33 AM, avi.e.gr...@gmail.com wrote:
> Thomas,
> 
> I have been thinking about the concept of being stingy with information as
> this is a fairly common occurrence when people ask for help. They often
ask
> for what they think they want while people like us keep asking why they
want
> that and perhaps offer guidance on how to get closer to what they NEED or
a
> better way.
> 
> In retrospect, Rich did give all the info he thought he needed. It boiled
> down to saying that he wants to 

Re: Extract lines from file, add to new files

[Mats Wichmann via Python-list]

On 2/3/24 10:58, Thomas Passin via Python-list wrote:
In my view this whole thread became murky and complicated because the OP 
did not write down the requirements for the program.  Requirements are 
needed to communicate with other people.  An individual may not need to 
actually write down the requirements - depending on their complexity - 
but they always exist even if only vaguely in a person's mind.  The 
requirements may include what tools or languages the person wants to use 
and why.


If you are asking for help, you need to communicate the requirements to 
the people you are asking for help from.


The OP may have thought the original post(s) contained enough of the 
requirements but as we know by now, they didn't.


The person asking for help may not realize they don't know enough to 
write down all the requirements; an effort to do so may bring that lack 
to visibility.


Mailing lists like these have a drawback that it's hard to impossible 
for someone not involved in a thread to learn anything general from it. 
We can write over and over again to please state clearly what you want 
to do and where the sticking points are, but newcomers post new 
questions without ever reading these pleas.  Then good-hearted people 
who want to be helpful end up spending a lot of time trying to guess 
what is actually being asked for, and maybe never find out with enough 
clarity.  Others take a guess and then spend time working up a solution 
that may or may not be on target.


So please! before posting a request for help, write down the 
requirements as best you can figure them out, and then make sure that 
they are expressed such that the readers can understand.


Indeed.  I've occasionally practised the following technique (in some 
form) over the years without knowing it had grown a name, and wikipedia 
page to go with it.  It may be handy to use to help come up with a 
clearer explanation before sending off a post to a mailing list or other 
static medium, because of the inevitable delays in going back and forth. 
Interactive formus like the Python Discord have a bit of an advantage in 
that you can try to tease out the intent more quickly.  But as you 
say... a newcomer won't know this.


https://en.wikipedia.org/wiki/Rubber_duck_debugging

--
https://mail.python.org/mailman/listinfo/python-list

Re: Extract lines from file, add to new files

[Thomas Passin via Python-list]

In my view this whole thread became murky and complicated because the OP 
did not write down the requirements for the program.  Requirements are 
needed to communicate with other people.  An individual may not need to 
actually write down the requirements - depending on their complexity - 
but they always exist even if only vaguely in a person's mind.  The 
requirements may include what tools or languages the person wants to use 
and why.


If you are asking for help, you need to communicate the requirements to 
the people you are asking for help from.


The OP may have thought the original post(s) contained enough of the 
requirements but as we know by now, they didn't.


The person asking for help may not realize they don't know enough to 
write down all the requirements; an effort to do so may bring that lack 
to visibility.


Mailing lists like these have a drawback that it's hard to impossible 
for someone not involved in a thread to learn anything general from it. 
We can write over and over again to please state clearly what you want 
to do and where the sticking points are, but newcomers post new 
questions without ever reading these pleas.  Then good-hearted people 
who want to be helpful end up spending a lot of time trying to guess 
what is actually being asked for, and maybe never find out with enough 
clarity.  Others take a guess and then spend time working up a solution 
that may or may not be on target.


So please! before posting a request for help, write down the 
requirements as best you can figure them out, and then make sure that 
they are expressed such that the readers can understand.


On 2/3/2024 11:33 AM, avi.e.gr...@gmail.com wrote:

Thomas,

I have been thinking about the concept of being stingy with information as
this is a fairly common occurrence when people ask for help. They often ask
for what they think they want while people like us keep asking why they want
that and perhaps offer guidance on how to get closer to what they NEED or a
better way.

In retrospect, Rich did give all the info he thought he needed. It boiled
down to saying that he wants to distribute data into two files in such a way
that finding an item in file A then lets him find the corresponding item in
file B. He was not worried about how to make the files or what to do with
the info afterward. He had those covered and was missing what he considered
a central piece. And, it seems he programs in multiple languages and
environments as needed and is not exactly a newbie. He just wanted a way to
implement his overall design.

We threw many solutions and ideas at him but some of us (like me) also got
frustrated as some ideas were not received due to one objection or another
that had not been mentioned earlier when it was not seen as important.

I particularly notice a disconnect some of us had. Was this supposed to be a
search that read only as much as needed to find something and stopped
reading, or a sort of filter that returned zero or more matches and went to
the end, or perhaps something that read entire files and swallowed them into
data structures in memory and then searched and found corresponding entries,
or maybe something else?

All the above approaches could work but some designs not so much. For
example, some files are too large. We, as programmers, often consciously or
unconsciously look at many factors to try to zoom in on what approaches me
might use. To be given minimal amounts of info can be frustrating. We worry
about making a silly design. But the OP may want something minimal and not
worry as long as it is fairly easy to program and works.

We could have suggested something very simple like:

Open both files A and B
In a loop get a line from each. If the line from A is a match, do something
with the current line from B.
If you are getting only one, exit the loop.

Or, if willing, we could have suggested any other file format, such as a
CSV, in which the algorithm is similar but different as in:

Open file A
Read a line in a loop
Split it in parts
If the party of the first part matches something, use the party of the
second part

Or, of course, suggest they read the entire file, into a list of lines or a
data.frame and use some tools that search all of it and produce results.

I find I personally now often lean toward the latter approach but ages ago
when memory and CPU were considerations and maybe garbage collection was not
automatic, ...


-Original Message-
From: Python-list  On
Behalf Of Thomas Passin via Python-list
Sent: Wednesday, January 31, 2024 7:25 AM
To: python-list@python.org
Subject: Re: Extract lines from file, add to new files

On 1/30/2024 11:25 PM, avi.e.gr...@gmail.com wrote:

Thomas, on some points we may see it differently.


I'm mostly going by what the OP originally asked for back on Jan 11.
He's been too stingy with information since then to be worth spending
much time on, IMHO.


Some formats can be done simply but are maybe better done in somewhat
standard 

RE: Extract lines from file, add to new files

[AVI GROSS via Python-list]

Thomas,

I have been thinking about the concept of being stingy with information as
this is a fairly common occurrence when people ask for help. They often ask
for what they think they want while people like us keep asking why they want
that and perhaps offer guidance on how to get closer to what they NEED or a
better way.

In retrospect, Rich did give all the info he thought he needed. It boiled
down to saying that he wants to distribute data into two files in such a way
that finding an item in file A then lets him find the corresponding item in
file B. He was not worried about how to make the files or what to do with
the info afterward. He had those covered and was missing what he considered
a central piece. And, it seems he programs in multiple languages and
environments as needed and is not exactly a newbie. He just wanted a way to
implement his overall design.

We threw many solutions and ideas at him but some of us (like me) also got
frustrated as some ideas were not received due to one objection or another
that had not been mentioned earlier when it was not seen as important.

I particularly notice a disconnect some of us had. Was this supposed to be a
search that read only as much as needed to find something and stopped
reading, or a sort of filter that returned zero or more matches and went to
the end, or perhaps something that read entire files and swallowed them into
data structures in memory and then searched and found corresponding entries,
or maybe something else?

All the above approaches could work but some designs not so much. For
example, some files are too large. We, as programmers, often consciously or
unconsciously look at many factors to try to zoom in on what approaches me
might use. To be given minimal amounts of info can be frustrating. We worry
about making a silly design. But the OP may want something minimal and not
worry as long as it is fairly easy to program and works.

We could have suggested something very simple like:

Open both files A and B
In a loop get a line from each. If the line from A is a match, do something
with the current line from B.
If you are getting only one, exit the loop.

Or, if willing, we could have suggested any other file format, such as a
CSV, in which the algorithm is similar but different as in:

Open file A
Read a line in a loop
Split it in parts
If the party of the first part matches something, use the party of the
second part

Or, of course, suggest they read the entire file, into a list of lines or a
data.frame and use some tools that search all of it and produce results.

I find I personally now often lean toward the latter approach but ages ago
when memory and CPU were considerations and maybe garbage collection was not
automatic, ...


-Original Message-
From: Python-list  On
Behalf Of Thomas Passin via Python-list
Sent: Wednesday, January 31, 2024 7:25 AM
To: python-list@python.org
Subject: Re: Extract lines from file, add to new files

On 1/30/2024 11:25 PM, avi.e.gr...@gmail.com wrote:
> Thomas, on some points we may see it differently.

I'm mostly going by what the OP originally asked for back on Jan 11. 
He's been too stingy with information since then to be worth spending 
much time on, IMHO.

> Some formats can be done simply but are maybe better done in somewhat
> standard ways.
> 
> Some of what the OP has is already tables in a database and that can
> trivially be exported into a CSV file or other formats like your TSV file
> and more. They can also import from there. As I mentioned, many
spreadsheets
> and all kinds of statistical programs tend to support some formats making
it
> quite flexible.
> 
> Python has all kinds of functionality, such as in the pandas module, to
read
> in a CSV or write it out. And once you have the data structure in memory,
al
> kinds of queries and changes can be made fairly straightforwardly. As one
> example, Rich has mentioned wanting finer control in selecting who gets
some
> version of the email based on concepts like market segmentation. He
already
> may have info like the STATE (as in Arizona) in his database. He might at
> some point enlarge his schema so each entry is placed in one or more
> categories and thus his CSV, once imported, can do the usual tasks of
> selecting various rows and columns or doing joins or whatever.
> 
> Mind you, another architecture could place quite a bit of work completely
on
> the back end and he could send SQL queries to the database from python and
> get back his results into python which would then make the email messages
> and pass them on to other functionality to deliver. This would remove any
> need for files and just rely on the DB.
> 
> There as as usual, too many choices and not necessarily one best answer.
Of
> course if this was a major product that would be heavily used, sure, you
> could tweak and optimize. As it is, Rich is getting a chance to improve
his
> python skills no matter which way he goes.
> 
> 
> 
> -Original Message-
> 

RE: Extract lines from file, add to new files

[AVI GROSS via Python-list]

This discussion has circled back to where it started. It illustrates quite a
few points about how many different ways someone can do something as well as
doing it using different tools and also about how others may see aspects of
mission creep as they look for making it perfect when it need not be. I mean
the OP wants a simple personal tool and is not currently concerned with lots
of things.

The data seems to be in a data base and can be extracted from there into one
or two files mostly for convenience. Presumably, all the usual
additions/deletions/modifications are being done in the DB. If so, then as
long as his latest vision of having two synchronized files is done
consistently by making both, they are in sync.

In addition, it sounds like the original idea was a tad too simple. The OP
wanted to add a single-line or so at the head of a message that simply
interpolates a name. This could be handled with all kinds of
form-letter-merge methods as well as search and replace methods of great
generality. But, for now, the OP is asking for something truly simple and
would be happy with a solution that is slow and clunky and uses a few
utilities in a shell script on LINUX. I took a look at doing this on bash
and ran into some surprises as this would have been trivial for me in a
shell like ksh. Setting variables is now done in a subshell which was done
why? All I know is if you could simply read a file one line at a time in a
loop and set one or more variables, then it becomes easy to use something
like combining an echo and a cat of the original message to make a message
with a variable heading and feed that to the mailer. And if the storage
format was CSV, again, you could easily peel it apart using something like
cut or awk or whatever.

The main reason for trying it in python may have been frustration with doing
it as a shell script. Fair enough. But the idea of two files may have been
made in the first place by the original idea of such a script. The question
posed was sort of how to search in one and then somehow find the
corresponding part of another. The mindset of the OP may have been focused
in a direction we were not clear on. As an example, there are ways to search
file A and get back a line number of the first match and then using perhaps
another utility you ask it to scan file B and just print out that line
number and nothing else.

The current solution the OP is considering sounds like a loop that goes
through both files in parallel. This would mean when one loop finds what it
is looking for, the other delivers what is in the same location. That may
work well enough for the purpose.

But I have learned over the years that having a few well-designed tools and
ideas and using them consistently, is often a good way to get results even
if it is not as imaginative and efficient as some other. If you are allowed
to design something from top to bottom, there may be very nice solutions.
But if asked to just do one small part in a design that already feels it has
solved other aspects, then suggesting they start over is often not taken
well. LOL!

I happen to think some storage format such as CSV would be a good idea IFF
doing this in Python for some reasons I mentioned earlier. I am not in love
with CSV and use other data formats as needed and some others would be fine
as long as your toolkit included some easy and reliable way to read the
contents in and manipulate to get your results. I do programming in multiple
languages and whether I am using python or something like R, I often end up
importing the data into some data.frame format, performing a pipeline of
manipulations using it and perhaps other such structures and generating
results and optionally saving some back to files. Tons of well-tested and
comfortable tools can be used if the data is set up right so even when it
can be easily done by hand, it may not be worth the effort as the standard
tools and tricks work and work fast enough.

Rich (AKA the OP) seems satisfied with having a solution. It may not be
ideal but for his needs, ...


-Original Message-
From: Python-list  On
Behalf Of Thomas Passin via Python-list
Sent: Wednesday, January 31, 2024 10:36 AM
To: python-list@python.org
Subject: Re: Extract lines from file, add to new files

On 1/31/2024 9:05 AM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
> 
>> If I had a script that's been working for 30 years, I'd probably just use
>> Python to do the personalizing and let the rest of the bash script do the
>> rest, like it always has. The Python program would pipe or send the
>> personalized messages to the rest of the bash program. Something in that
>> ballpark, anyway.
> 
> Thomas,
> 
> A bash shell script looks easier for me and more promising. Using a while
> loop (one for the name file the other for the address file), and sed for
> putting the name at the head of the message replacing a generic
placeholder
> should work with the