Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Ethan Furman via Python-list]

This thread is derailing.

Please consider it closed.

--
~Ethan~
Moderator
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Left Right via Python-list]

> You can't validate an IP packet without having all of it. Your notion
> of "streaming" is nonsensical.

Whoa, whoa, hold your horses! "nonsensical" needs a little bit of
justification :)

It seems you don't understand the difference between words and
languages! In my examples, IP _protocol_ is the language, sequences of
IP packets are the words in the language. A language is amenable to
streaming if the words of the language are repetition of sequences of
symbols of the alphabet of fixed length.  This is, essentially, like
saying that the words themselves are regular.

So, the follow-up question from you to me should be: how come strictly
context-free languages can still be parsed with streaming parsers? --
And the answer to that is it's possible to approximate context-free
languages with regular languages.  In fact, this is a very interesting
subject, which unfortunately is usually overlooked in automata
classes.  It's interesting in a sense that it's very accessible to the
students who already mastered the understanding of regular and
context-free formalisms.

So, streaming parsers (eg. SAX) are written for a regular language
that approximates XML.  This is because in practice we will almost
never encounter more than N nesting levels in an XML, more than N
characters in an element name etc. (for some large enough N).
Something which allows us to create a regular language from a
context-free one.

NB. "Nonsensical" has a very precise meaning, when it comes to
discussing the truth value of a proposition, which I think you also
somehow didn't know about.  You seem to use "nonsensical" as a synonym
to "wrong".  But, unbeknownst to you, you said something else.  You
actually implied that there's no way to tell if my notion of streaming
is correct or not.

But, for the future reference: my notion of streaming is correct, and
you would do better learning some materials about it before jumping to
conclusions.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Left Right via Python-list]

> One single IP packet is all you can parse.

I worked for an undisclosed company which manufactures h/w for ISPs
(4- and 8-unit boxes you mount on a rack in a datacenter).
Essentially, big-big routers.  So, I had the pleasure of writing
software that parses IP _protocol_, and let me tell you: you have no
idea what you just wrote.

But, like I wrote earlier: you don't understand the distinction
between languages and words.  And in general, are just being stubborn
and rude because you are trying to prove a point to someone you don't
like, but, in reality, you just look more and more ridiculous.

On Thu, Oct 3, 2024 at 12:51 AM Chris Angelico  wrote:
>
> On Thu, 3 Oct 2024 at 08:48, Left Right  wrote:
> >
> > > You can't validate an IP packet without having all of it. Your notion
> > > of "streaming" is nonsensical.
> >
> > Whoa, whoa, hold your horses! "nonsensical" needs a little bit of
> > justification :)
> >
> > It seems you don't understand the difference between words and
> > languages! In my examples, IP _protocol_ is the language, sequences of
> > IP packets are the words in the language. A language is amenable to
> > streaming if the words of the language are repetition of sequences of
> > symbols of the alphabet of fixed length.  This is, essentially, like
> > saying that the words themselves are regular.
>
> One single IP packet is all you can parse. You're playing shenanigans
> with words the way Humpty Dumpty does. IP packets are not sequences,
> they are individuals.
>
> ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Chris Angelico via Python-list]

On Thu, 3 Oct 2024 at 08:48, Left Right  wrote:
>
> > You can't validate an IP packet without having all of it. Your notion
> > of "streaming" is nonsensical.
>
> Whoa, whoa, hold your horses! "nonsensical" needs a little bit of
> justification :)
>
> It seems you don't understand the difference between words and
> languages! In my examples, IP _protocol_ is the language, sequences of
> IP packets are the words in the language. A language is amenable to
> streaming if the words of the language are repetition of sequences of
> symbols of the alphabet of fixed length.  This is, essentially, like
> saying that the words themselves are regular.

One single IP packet is all you can parse. You're playing shenanigans
with words the way Humpty Dumpty does. IP packets are not sequences,
they are individuals.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Chris Angelico via Python-list]

On Wed, 2 Oct 2024 at 23:53, Left Right via Python-list
 wrote:
> In the same email you replied to, I gave examples of languages for
> which parsers can be streaming (in general): SCSI or IP.

You can't validate an IP packet without having all of it. Your notion
of "streaming" is nonsensical.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Left Right via Python-list]

> By that definition of "streaming", no parser can ever be streaming,
> because there will be some constructs that must be read in their
> entirety before a suitably-structured piece of output can be
> emitted.

In the same email you replied to, I gave examples of languages for
which parsers can be streaming (in general): SCSI or IP. For some
languages (eg. everything in the context-free family) streaming
parsers are _in general_ impossible, because there are pathological
cases like the one with parsing numbers. But this doesn't mean that
you cannot come up with a parser that is only useful _sometimes_.
And, in practice, languages like XML or JSON do well with streaming,
even though in general it's impossible.

I'm sorry if this comes as a surprise.  On one hand I don't want to
sound condescending, on the other hand, this is something that you'd
typically study in automata theory class.  Well, not exactly in the
very same words, but you should be able to figure this stuff out if
you had that class.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Greg Ewing via Python-list]

On 2/10/24 12:26 pm, avi.e.gr...@gmail.com wrote:

The real problem is how the JSON is set up. If you take umpteen data
structures and wrap them all in something like a list, then it may be a tad
hard to stream as you may not necessarily be examining the contents till the
list finishes gigabytes later.


Yes, if you want to process the items as they come in, you might
be better off sending a series of separate JSON strings, rather than
one JSON string containing a list.

Or, use a specialised JSON parser that processes each item of the
list as soon as it's finished parsing it, instead of collecting the
whole list first.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Dan Sommers via Python-list]

On 2024-10-01 at 23:03:01 +0200,
Left Right  wrote:

> > If I recognize the first digit, then I *can* hand that over to an
> > external function to accumulate the digits that follow.
> 
> And what is that external function going to do with this information?
> The point is you didn't parse anything if you just sent the digit.
> You just delegated the parsing further. Parsing is only meaningful if
> you extracted some information, but your idea is, essentially "what if
> I do nothing?".

If the parser detects the first digit of a number, then the parser can
read digits one at a time (i.e., "streaming"), assimilate and accumulate
the value of the number being parsed, and successfully finish parsing
the number it reads a non-digit.  Whether the function that accumulates
the value during the process is internal or external isn't relevant; the
point is that it is possible to parse integers from most significant
digit to least significant digit under a streaming model (and if you're
sufficiently clever, you can even write partial results to external
storage and/or another transmission protocol, thus allowing for numbers
bigger (as measured by JSON or your internal representation) than your
RAM).

At most, the parser has to remember the non-digit character it read so
that it (the parser) can begin to parse whatever comes after the number.
Does that break your notion of "streaming"?

Why do I have to start with the least significant digit?

> > Under that constraint, I'm not sure I can parse anything.  How can I
> > parse a string (and hand it over to an external function) until I've
> > found the closing quote?
> 
> Nobody says that parsing a number is the only pathological case.  You,
> however, exaggerate by saying you cannot parse _anything_. You can
> parse booleans or null, for example.  There's no problem there.

My intent was only to repeat what you implied:  that any parser that
reads its input until it has parsed a value is not streaming.

So how much information can the parser keep before you consider it not
to be "streaming"?

[...]

> In principle, any language that has infinite words will have the same
> problem with streaming [...]

So what magic allows anyone to stream any JSON file over SCSI or IP?
Let alone some kind of "live stream" that by definition is indefinite,
even if it only lasts a few tenths of a second?

> [...] If you ever pondered h/w or low-level
> protocols s.a. SCSI or IP [...]

I spent a good deal of my career designing and implementing all manner
of communicaations protocols, from transmitting and receiving single
bits over a wire all the way up to what are now known as session and
presentation layers.  Some imposed maximum lengths in certain places;
some allowed for indefinite amounts of data to be transferred from one
end to the other without stopping, resetting, or overflowing.  And yet
somehow, the universe never collapsed.

If you believe that some implementation of fsync fails to meet a
specification, or fails to work correctly on files containign JSON, then
file a bug report.
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[AVI GROSS via Python-list]

This discussion has become less useful.

E can all agree that in Computer Science, real infinities are avoided, and
frankly, need not be taken seriously in any serious program.

You can store all kinds of infinities quite compactly as in a transcendental
number you can derive to as many decimal points as you like. Want 1/7 to a
thousand decimal places, no problem. You can be given a digit 1 and a digit
7 and asked to do a division to as many digits as you wish in a
deterministic manner. I can think of quite a few generators that could
easily supply the next digit, or just keep giving the next element from
142857 each time from a circular loop.

Sines, cosines, pi, e and so on, can often be calculated to arbitrary
precision by evaluating things like infinite Taylor Series as many times as
needed up to the precision of the data holding the number as you move along.

 Similar ideas allow generators to give you as many primes as you want, and
no more.

So, if you can store arbitrary python code as part of your JSON, you can
send quite a bit of somewhat compressed data.

The real problem is how the JSON is set up. If you take umpteen data
structures and wrap them all in something like a list, then it may be a tad
hard to stream as you may not necessarily be examining the contents till the
list finishes gigabytes later. But if, instead, you send lots of smaller
parts, such as perhaps sending each row of something like a data.frame
individually, the other side can recombine them incrementally to a larger
structure such as a data.frame and do some logic on it as it streams, such
as keeping only some columns and discarding the rest, or applying filters
that only keep rows you care about. And, of course, all rows could be
appended to one and perhaps more .CSV files as well so if you need multiple
passes on the data, it can now be processed locally in various modes,
including "streamed".

I think that for some purposes, it makes some sense to not stream anything
but results. I mean consider any database that allows a remote login and SQL
commands that only stream results. If I only want info on records about
company X between July 1 and September 15 of a particular year and only if
the amount paid remains zero or is less than the amount owed, ...


-Original Message-
From: Python-list  On
Behalf Of Greg Ewing via Python-list
Sent: Tuesday, October 1, 2024 5:48 PM
To: python-list@python.org
Subject: Re: Help with Streaming and Chunk Processing for Large JSON Data
(60 GB) from Kenna API

On 1/10/24 8:34 am, Left Right wrote:
> You probably forgot that it has to be _streaming_. Suppose you parse
> the first digit: can you hand this information over to an external
> function to process the parsed data? -- No! because you don't know the
> magnitude yet.

By that definition of "streaming", no parser can ever be streaming,
because there will be some constructs that must be read in their
entirety before a suitably-structured piece of output can be
emitted.

The context of this discussion about integers is the claim that
they *could* be parsed incrementally if they were written little
endian instead of big endian, but the same argument applies either
way.

-- 
Greg
-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Greg Ewing via Python-list]

On 2/10/24 10:03 am, Left Right wrote:

Consider also an interesting
consequence of SCSI not being able to have infinite words: this means,
besides other things that fsync() is nonsense! :) If you aren't
familiar with the concept: UNIX filesystem API suggests that it's
possible to destage arbitrary large file (or a chunk of file) to disk.
But SCSI is built of finite "words" and to describe an arbitrary large
file you'd need to list all the blocks that constitute the file!


I don't follow. What fsync() does is ensure that any data buffered
in the kernel relating to the file is sent to the storage device.
It can send as many blocks of data over SCSI as required to
achieve this. There's no requirement for it to be atomic at the
level of the interface between the kernel and the hardware.

Some devices do their own buffering in ways that are invisible to
the software, so fsync() can't guarantee that the data is actually
written to the storage medium. But that's a problem stemming from
the design of the hardware, not the design of the protocol for
communicating with the hardware.

> the only way to implement fsync() in compliance with the
> standard is to sync _everything_

Again I'm not sure what you mean here. It may be difficult for the
kernel to track down exactly what data is relevant to a particular file,
and so the kernel programmers take the easy way out and just implement
fsync() as sync(). But again that has nothing to do with the protocol.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Greg Ewing via Python-list]

On 1/10/24 8:34 am, Left Right wrote:

You probably forgot that it has to be _streaming_. Suppose you parse
the first digit: can you hand this information over to an external
function to process the parsed data? -- No! because you don't know the
magnitude yet.


By that definition of "streaming", no parser can ever be streaming,
because there will be some constructs that must be read in their
entirety before a suitably-structured piece of output can be
emitted.

The context of this discussion about integers is the claim that
they *could* be parsed incrementally if they were written little
endian instead of big endian, but the same argument applies either
way.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Left Right via Python-list]

> If I recognize the first digit, then I *can* hand that over to an
> external function to accumulate the digits that follow.

And what is that external function going to do with this information?
The point is you didn't parse anything if you just sent the digit.
You just delegated the parsing further. Parsing is only meaningful if
you extracted some information, but your idea is, essentially "what if
I do nothing?".

> Under that constraint, I'm not sure I can parse anything.  How can I
parse a string (and hand it over to an external function) until I've
found the closing quote?

Nobody says that parsing a number is the only pathological case.  You,
however, exaggerate by saying you cannot parse _anything_. You can
parse booleans or null, for example.  There's no problem there.

Again, I think you misunderstand what streaming is for. Let me remind:
it's for processing information as it comes, potentially,
indefinitely. This has far more important implications than what you
find in computer science. For example, some mathematicians use the
same argument to show that real numbers are either fiction or useless:
consider adding two real numbers (where real numbers are potentially
infinite strings of decimal digits after the period) -- there's no way
to prove that such an addition is possible because you would need an
infinite proof for that (because you need to start adding from the
least significant digit).

In principle, any language that has infinite words will have the same
problem with streaming. If you ever pondered h/w or low-level
protocols s.a. SCSI or IP, you'd see that they are specifically
designed in such a way as to never have infinite words (because they
must be amenable to streaming). Consider also an interesting
consequence of SCSI not being able to have infinite words: this means,
besides other things that fsync() is nonsense! :) If you aren't
familiar with the concept: UNIX filesystem API suggests that it's
possible to destage arbitrary large file (or a chunk of file) to disk.
But SCSI is built of finite "words" and to describe an arbitrary large
file you'd need to list all the blocks that constitute the file!  And
that's why fsync() and family are so hated by people who deal with
storage: the only way to implement fsync() in compliance with the
standard is to sync _everything_ (and it hurts!)

On Tue, Oct 1, 2024 at 5:49 PM Dan Sommers via Python-list
 wrote:
>
> On 2024-09-30 at 21:34:07 +0200,
> Regarding "Re: Help with Streaming and Chunk Processing for Large JSON Data 
> (60 GB) from Kenna API,"
> Left Right via Python-list  wrote:
>
> > > What am I missing?  Handwavingly, start with the first digit, and as
> > > long as the next character is a digit, multipliy the accumulated result
> > > by 10 (or the appropriate base) and add the next value.  Oh, and handle
> > > scientific notation as a special case, and perhaps fail spectacularly
> > > instead of recovering gracefully in certain edge cases.  And in the
> > > pathological case of a single number with 60 billion digits, run out of
> > > memory (and complain loudly to the person who claimed that the file
> > > contained a "dataset").  But why do I need to start with the least
> > > significant digit?
> >
> > You probably forgot that it has to be _streaming_. Suppose you parse
> > the first digit: can you hand this information over to an external
> > function to process the parsed data? -- No! because you don't know the
> > magnitude yet.  What about two digits? -- Same thing.  You cannot
> > leave the parser code until you know the magnitude (otherwise the
> > information is useless to the external code).
>
> If I recognize the first digit, then I *can* hand that over to an
> external function to accumulate the digits that follow.
>
> > So, even if you have enough memory and don't care about special cases
> > like scientific notation: yes, you will be able to parse it, but it
> > won't be a streaming parser.
>
> Under that constraint, I'm not sure I can parse anything.  How can I
> parse a string (and hand it over to an external function) until I've
> found the closing quote?
>
> How much state can a parser maintain (before it invokes an external
> function) and still be considered streaming?  I fear that we may be
> getting hung up on terminology rather than solving the problem at hand.
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Dan Sommers via Python-list]

On 2024-09-30 at 21:34:07 +0200,
Regarding "Re: Help with Streaming and Chunk Processing for Large JSON Data (60 
GB) from Kenna API,"
Left Right via Python-list  wrote:

> > What am I missing?  Handwavingly, start with the first digit, and as
> > long as the next character is a digit, multipliy the accumulated result
> > by 10 (or the appropriate base) and add the next value.  Oh, and handle
> > scientific notation as a special case, and perhaps fail spectacularly
> > instead of recovering gracefully in certain edge cases.  And in the
> > pathological case of a single number with 60 billion digits, run out of
> > memory (and complain loudly to the person who claimed that the file
> > contained a "dataset").  But why do I need to start with the least
> > significant digit?
> 
> You probably forgot that it has to be _streaming_. Suppose you parse
> the first digit: can you hand this information over to an external
> function to process the parsed data? -- No! because you don't know the
> magnitude yet.  What about two digits? -- Same thing.  You cannot
> leave the parser code until you know the magnitude (otherwise the
> information is useless to the external code).

If I recognize the first digit, then I *can* hand that over to an
external function to accumulate the digits that follow.

> So, even if you have enough memory and don't care about special cases
> like scientific notation: yes, you will be able to parse it, but it
> won't be a streaming parser.

Under that constraint, I'm not sure I can parse anything.  How can I
parse a string (and hand it over to an external function) until I've
found the closing quote?

How much state can a parser maintain (before it invokes an external
function) and still be considered streaming?  I fear that we may be
getting hung up on terminology rather than solving the problem at hand.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Dan Sommers via Python-list]

On 2024-09-30 at 18:48:02 -0700,
Keith Thompson via Python-list  wrote:

> 2qdxy4rzwzuui...@potatochowder.com writes:
> [...]
> > In Common Lisp, you can write integers as #nnR[digits], where nn is the
> > decimal representation of the base (possibly without a leading zero),
> > the # and the R are literal characters, and the digits are written in
> > the intended base.  So the input #16f is read as the integer 65535.
> 
> Typo: You meant #16R, not #16f.

Yep.  Sorry.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Left Right via Python-list]

> What am I missing?  Handwavingly, start with the first digit, and as
> long as the next character is a digit, multipliy the accumulated result
> by 10 (or the appropriate base) and add the next value.  Oh, and handle
> scientific notation as a special case, and perhaps fail spectacularly
> instead of recovering gracefully in certain edge cases.  And in the
> pathological case of a single number with 60 billion digits, run out of
> memory (and complain loudly to the person who claimed that the file
> contained a "dataset").  But why do I need to start with the least
> significant digit?

You probably forgot that it has to be _streaming_. Suppose you parse
the first digit: can you hand this information over to an external
function to process the parsed data? -- No! because you don't know the
magnitude yet.  What about two digits? -- Same thing.  You cannot
leave the parser code until you know the magnitude (otherwise the
information is useless to the external code).

So, even if you have enough memory and don't care about special cases
like scientific notation: yes, you will be able to parse it, but it
won't be a streaming parser.

On Mon, Sep 30, 2024 at 9:30 PM Left Right  wrote:
>
> > Streaming won't work because the file is gzipped.  You have to receive
> > the whole thing before you can unzip it. Once unzipped it will be even
> > larger, and all in memory.
>
> GZip is specifically designed to be streamed.  So, that's not a
> problem (in principle), but you would need to have a streaming GZip
> parser, quick search in PyPI revealed this package:
> https://pypi.org/project/gzip-stream/ .
>
> On Mon, Sep 30, 2024 at 6:20 PM Thomas Passin via Python-list
>  wrote:
> >
> > On 9/30/2024 11:30 AM, Barry via Python-list wrote:
> > >
> > >
> > >> On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list 
> > >>  wrote:
> > >>
> > >>
> > >> import polars as pl
> > >> pl.read_json("file.json")
> > >>
> > >>
> > >
> > > This is not going to work unless the computer has a lot more the 60GiB of 
> > > RAM.
> > >
> > > As later suggested a streaming parser is required.
> >
> > Streaming won't work because the file is gzipped.  You have to receive
> > the whole thing before you can unzip it. Once unzipped it will be even
> > larger, and all in memory.
> > --
> > https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Keith Thompson via Python-list]

2qdxy4rzwzuui...@potatochowder.com writes:
[...]
> In Common Lisp, you can write integers as #nnR[digits], where nn is the
> decimal representation of the base (possibly without a leading zero),
> the # and the R are literal characters, and the digits are written in
> the intended base.  So the input #16f is read as the integer 65535.

Typo: You meant #16R, not #16f.

-- 
Keith Thompson (The_Other_Keith) keith.s.thompso...@gmail.com
void Void(void) { Void(); } /* The recursive call of the void */
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Dan Sommers via Python-list]

On 2024-10-01 at 09:09:07 +1000,
Chris Angelico via Python-list  wrote:

> On Tue, 1 Oct 2024 at 08:56, Grant Edwards via Python-list
>  wrote:
> >
> > On 2024-09-30, Dan Sommers via Python-list  wrote:
> >
> > > In Common Lisp, integers can be written in any integer base from two
> > > to thirty six, inclusive.  So knowing the last digit doesn't tell
> > > you whether an integer is even or odd until you know the base
> > > anyway.
> >
> > I had to think about that for an embarassingly long time before it
> > clicked.
> 
> The only part I'm not clear on is what identifies the base. If you're
> going to write numbers little-endian, it's not that hard to also write
> them with a base indicator before the digits [...]

In Common Lisp, you can write integers as #nnR[digits], where nn is the
decimal representation of the base (possibly without a leading zero),
the # and the R are literal characters, and the digits are written in
the intended base.  So the input #16f is read as the integer 65535.

You can also set or bind the global variable *read-base* (yes, the
asterisks are part of the name) to an integer between 2 and 36, and then
anything that looks like an integer in that base is interpreted as such
(including literals in programs).  The literals I described above are
still handled correctly no matter the current value of *read-base*.  So
if the value of *read-base* is 16, then the input  is read as the
integer 65535 (as is the input #16r).

(Pedants may point our details I omitted.  I admit to omitting them.)

IIRC, certain [old 8080 and Z-80?] assemblers used to put the base
indicator at the end.  So 10 meant, well, 10, but 10H meant 16 and 10b
meant 2 (IDK; the capital H and the lower case b both look right to me).

I don't recall numbers written from least significant digit to most
significant digit (big and little endian *storage*, yes, but not the
digits when presented to or read from a human).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Chris Angelico via Python-list]

On Tue, 1 Oct 2024 at 08:56, Grant Edwards via Python-list
 wrote:
>
> On 2024-09-30, Dan Sommers via Python-list  wrote:
>
> > In Common Lisp, integers can be written in any integer base from two
> > to thirty six, inclusive.  So knowing the last digit doesn't tell
> > you whether an integer is even or odd until you know the base
> > anyway.
>
> I had to think about that for an embarassingly long time before it
> clicked.

The only part I'm not clear on is what identifies the base. If you're
going to write numbers little-endian, it's not that hard to also write
them with a base indicator before the digits. But, whatever. This is a
typical tangent and people are argumentative for no reason. I was just
trying to add some explanatory notes to why little-endian does make
more sense than big-endian.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Grant Edwards via Python-list]

On 2024-09-30, Dan Sommers via Python-list  wrote:

> In Common Lisp, integers can be written in any integer base from two
> to thirty six, inclusive.  So knowing the last digit doesn't tell
> you whether an integer is even or odd until you know the base
> anyway.

I had to think about that for an embarassingly long time before it
clicked.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Dan Sommers via Python-list]

On 2024-10-01 at 04:46:35 +1000,
Chris Angelico via Python-list  wrote:

> On Tue, 1 Oct 2024 at 04:30, Dan Sommers via Python-list
>  wrote:
> >
> > But why do I need to start with the least
> > significant digit?
> 
> If you start from the most significant, you don't know anything about
> the number until you finish parsing it. There's almost nothing you can
> say about a number given that it starts with a particular sequence
> (since you don't know how MANY digits there are). However, if you know
> the LAST digits, you can make certain statements about it (trivial
> examples being whether it's odd or even).

But that wasn't the question.  Sure, under certain circumstances and for
specific use cases and/or requirements, there might be arguments to read
potential numbers as strings and possibly not have to parse them
completely before accepting or rejecting them.

And if I start with the least significant digit and the number happens
to be written in scientific notation and/or has a decimal point, then I
can't tell whether it's odd or even until I further process the whole
thing anyway.

> It's not very, well, significant. But there's something to it. And it
> extends nicely to p-adic numbers, which can have an infinite number of
> nonzero digits to the left of the decimal:
> 
> https://en.wikipedia.org/wiki/P-adic_number

In Common Lisp, integers can be written in any integer base from two to
thirty six, inclusive.  So knowing the last digit doesn't tell you
whether an integer is even or odd until you know the base anyway.

Curiously, we agree:  if you move the goal posts arbitrarily, then
some algorithms that parse JSON numbers will fail.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Thomas Passin via Python-list]

On 9/30/2024 11:30 AM, Barry via Python-list wrote:




On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list 
 wrote:


import polars as pl
pl.read_json("file.json")




This is not going to work unless the computer has a lot more the 60GiB of RAM.

As later suggested a streaming parser is required.


There is also the json-stream library, on PyPi at

https://pypi.org/project/json-stream/


--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Left Right via Python-list]

> Streaming won't work because the file is gzipped.  You have to receive
> the whole thing before you can unzip it. Once unzipped it will be even
> larger, and all in memory.

GZip is specifically designed to be streamed.  So, that's not a
problem (in principle), but you would need to have a streaming GZip
parser, quick search in PyPI revealed this package:
https://pypi.org/project/gzip-stream/ .

On Mon, Sep 30, 2024 at 6:20 PM Thomas Passin via Python-list
 wrote:
>
> On 9/30/2024 11:30 AM, Barry via Python-list wrote:
> >
> >
> >> On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list 
> >>  wrote:
> >>
> >>
> >> import polars as pl
> >> pl.read_json("file.json")
> >>
> >>
> >
> > This is not going to work unless the computer has a lot more the 60GiB of 
> > RAM.
> >
> > As later suggested a streaming parser is required.
>
> Streaming won't work because the file is gzipped.  You have to receive
> the whole thing before you can unzip it. Once unzipped it will be even
> larger, and all in memory.
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Thomas Passin via Python-list]

On 9/30/2024 1:00 PM, Chris Angelico via Python-list wrote:

On Tue, 1 Oct 2024 at 02:20, Thomas Passin via Python-list
 wrote:


On 9/30/2024 11:30 AM, Barry via Python-list wrote:




On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list 
 wrote:


import polars as pl
pl.read_json("file.json")




This is not going to work unless the computer has a lot more the 60GiB of RAM.

As later suggested a streaming parser is required.


Streaming won't work because the file is gzipped.  You have to receive
the whole thing before you can unzip it. Once unzipped it will be even
larger, and all in memory.


Streaming gzip is perfectly possible. You may be thinking of PKZip
which has its EOCD at the end of the file (although it may still be
possible to stream-decompress if you work at it).

ChrisA


You're right, that's what I was thinking of.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Chris Angelico via Python-list]

On Tue, 1 Oct 2024 at 04:30, Dan Sommers via Python-list
 wrote:
>
> But why do I need to start with the least
> significant digit?

If you start from the most significant, you don't know anything about
the number until you finish parsing it. There's almost nothing you can
say about a number given that it starts with a particular sequence
(since you don't know how MANY digits there are). However, if you know
the LAST digits, you can make certain statements about it (trivial
examples being whether it's odd or even).

It's not very, well, significant. But there's something to it. And it
extends nicely to p-adic numbers, which can have an infinite number of
nonzero digits to the left of the decimal:

https://en.wikipedia.org/wiki/P-adic_number

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Grant Edwards via Python-list]

On 2024-09-30, Dan Sommers via Python-list  wrote:
> On 2024-09-30 at 11:44:50 -0400,
> Grant Edwards via Python-list  wrote:
>
>> On 2024-09-30, Left Right via Python-list  wrote:
>> > [...]
>> > Imagine a pathological case of this shape: 1... <60GB of digits>. This
>> > is still a valid JSON (it doesn't have any limits on how many digits a
>> > number can have). And you cannot parse this number in a streaming way
>> > because in order to do that, you need to start with the least
>> > significant digit.
>> 
>> Which is how arabic numbers were originally parsed, but when
>> westerners adopted them from a R->L written language, thet didn't
>> flip them around to match the L->R written language into which they
>> were being adopted.
>
> Interesting.
>
>> So now long numbers can't be parsed as a stream in software. They
>> should have anticipated this problem back in the 13th century and
>> flipped the numbers around.
>
> What am I missing?  Handwavingly, start with the first digit, and as
> long as the next character is a digit, multipliy the accumulated
> result by 10 (or the appropriate base) and add the next value.
> [...]  But why do I need to start with the least significant digit?

Excellent question.  That's actully a pretty standard way to parse
numeric literals. I accepted the claim at face value that in JSON
there is something that requires parsing numeric literals from the
least significant end -- but I can't think of why the usual algorithms
used by other languages' lexers for yonks wouldn't work for JSON.

--
Grant
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Dan Sommers via Python-list]

On 2024-09-30 at 11:44:50 -0400,
Grant Edwards via Python-list  wrote:

> On 2024-09-30, Left Right via Python-list  wrote:
> > Whether and to what degree you can stream JSON depends on JSON
> > structure. In general, however, JSON cannot be streamed (but commonly
> > it can be).
> >
> > Imagine a pathological case of this shape: 1... <60GB of digits>. This
> > is still a valid JSON (it doesn't have any limits on how many digits a
> > number can have). And you cannot parse this number in a streaming way
> > because in order to do that, you need to start with the least
> > significant digit.
> 
> Which is how arabic numbers were originally parsed, but when
> westerners adopted them from a R->L written language, thet didn't flip
> them around to match the L->R written language into which they were
> being adopted.

Interesting.

> So now long numbers can't be parsed as a stream in software. They
> should have anticipated this problem back in the 13th century and
> flipped the numbers around.

What am I missing?  Handwavingly, start with the first digit, and as
long as the next character is a digit, multipliy the accumulated result
by 10 (or the appropriate base) and add the next value.  Oh, and handle
scientific notation as a special case, and perhaps fail spectacularly
instead of recovering gracefully in certain edge cases.  And in the
pathological case of a single number with 60 billion digits, run out of
memory (and complain loudly to the person who claimed that the file
contained a "dataset").  But why do I need to start with the least
significant digit?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Chris Angelico via Python-list]

On Tue, 1 Oct 2024 at 02:20, Thomas Passin via Python-list
 wrote:
>
> On 9/30/2024 11:30 AM, Barry via Python-list wrote:
> >
> >
> >> On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list 
> >>  wrote:
> >>
> >>
> >> import polars as pl
> >> pl.read_json("file.json")
> >>
> >>
> >
> > This is not going to work unless the computer has a lot more the 60GiB of 
> > RAM.
> >
> > As later suggested a streaming parser is required.
>
> Streaming won't work because the file is gzipped.  You have to receive
> the whole thing before you can unzip it. Once unzipped it will be even
> larger, and all in memory.

Streaming gzip is perfectly possible. You may be thinking of PKZip
which has its EOCD at the end of the file (although it may still be
possible to stream-decompress if you work at it).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Thomas Passin via Python-list]

On 9/30/2024 11:30 AM, Barry via Python-list wrote:




On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list 
 wrote:


import polars as pl
pl.read_json("file.json")




This is not going to work unless the computer has a lot more the 60GiB of RAM.

As later suggested a streaming parser is required.


Streaming won't work because the file is gzipped.  You have to receive 
the whole thing before you can unzip it. Once unzipped it will be even 
larger, and all in memory.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Grant Edwards via Python-list]

On 2024-09-30, Left Right via Python-list  wrote:
> Whether and to what degree you can stream JSON depends on JSON
> structure. In general, however, JSON cannot be streamed (but commonly
> it can be).
>
> Imagine a pathological case of this shape: 1... <60GB of digits>. This
> is still a valid JSON (it doesn't have any limits on how many digits a
> number can have). And you cannot parse this number in a streaming way
> because in order to do that, you need to start with the least
> significant digit.

Which is how arabic numbers were originally parsed, but when
westerners adopted them from a R->L written language, thet didn't flip
them around to match the L->R written language into which they were
being adopted.

So now long numbers can't be parsed as a stream in software. They
should have anticipated this problem back in the 13th century and
flipped the numbers around.




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Barry via Python-list]

> On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list 
>  wrote:
> 
> 
> import polars as pl
> pl.read_json("file.json")
> 
> 

This is not going to work unless the computer has a lot more the 60GiB of RAM.

As later suggested a streaming parser is required.

Barry


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Left Right via Python-list]

Whether and to what degree you can stream JSON depends on JSON
structure. In general, however, JSON cannot be streamed (but commonly
it can be).

Imagine a pathological case of this shape: 1... <60GB of digits>. This
is still a valid JSON (it doesn't have any limits on how many digits a
number can have). And you cannot parse this number in a streaming way
because in order to do that, you need to start with the least
significant digit.

Typically, however, JSON can be parsed incrementally. The format is
conceptually very simple to write a parser for. There are plenty of
parsers that do that, for example, this one:
https://pypi.org/project/json-stream/ . But, I'd encourage you to do
it yourself.  It's fun, and the resulting parser should end up less
than some 50 LoC.  Also, it allows you to closer incorporate your
desired output into your parser.

On Mon, Sep 30, 2024 at 8:44 AM Asif Ali Hirekumbi via Python-list
 wrote:
>
> Thanks Abdur Rahmaan.
> I will give it a try !
>
> Thanks
> Asif
>
> On Mon, Sep 30, 2024 at 11:19 AM Abdur-Rahmaan Janhangeer <
> arj.pyt...@gmail.com> wrote:
>
> > Idk if you tried Polars, but it seems to work well with JSON data
> >
> > import polars as pl
> > pl.read_json("file.json")
> >
> > Kind Regards,
> >
> > Abdur-Rahmaan Janhangeer
> > about  | blog
> > 
> > github 
> > Mauritius
> >
> >
> > On Mon, Sep 30, 2024 at 8:00 AM Asif Ali Hirekumbi via Python-list <
> > python-list@python.org> wrote:
> >
> >> Dear Python Experts,
> >>
> >> I am working with the Kenna Application's API to retrieve vulnerability
> >> data. The API endpoint provides a single, massive JSON file in gzip
> >> format,
> >> approximately 60 GB in size. Handling such a large dataset in one go is
> >> proving to be quite challenging, especially in terms of memory management.
> >>
> >> I am looking for guidance on how to efficiently stream this data and
> >> process it in chunks using Python. Specifically, I am wondering if there’s
> >> a way to use the requests library or any other libraries that would allow
> >> us to pull data from the API endpoint in a memory-efficient manner.
> >>
> >> Here are the relevant API endpoints from Kenna:
> >>
> >>- Kenna API Documentation
> >>
> >>- Kenna Vulnerabilities Export
> >>
> >>
> >> If anyone has experience with similar use cases or can offer any advice,
> >> it
> >> would be greatly appreciated.
> >>
> >> Thank you in advance for your help!
> >>
> >> Best regards
> >> Asif Ali
> >> --
> >> https://mail.python.org/mailman/listinfo/python-list
> >>
> >
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Asif Ali Hirekumbi via Python-list]

Thanks Abdur Rahmaan.
I will give it a try !

Thanks
Asif

On Mon, Sep 30, 2024 at 11:19 AM Abdur-Rahmaan Janhangeer <
arj.pyt...@gmail.com> wrote:

> Idk if you tried Polars, but it seems to work well with JSON data
>
> import polars as pl
> pl.read_json("file.json")
>
> Kind Regards,
>
> Abdur-Rahmaan Janhangeer
> about  | blog
> 
> github 
> Mauritius
>
>
> On Mon, Sep 30, 2024 at 8:00 AM Asif Ali Hirekumbi via Python-list <
> python-list@python.org> wrote:
>
>> Dear Python Experts,
>>
>> I am working with the Kenna Application's API to retrieve vulnerability
>> data. The API endpoint provides a single, massive JSON file in gzip
>> format,
>> approximately 60 GB in size. Handling such a large dataset in one go is
>> proving to be quite challenging, especially in terms of memory management.
>>
>> I am looking for guidance on how to efficiently stream this data and
>> process it in chunks using Python. Specifically, I am wondering if there’s
>> a way to use the requests library or any other libraries that would allow
>> us to pull data from the API endpoint in a memory-efficient manner.
>>
>> Here are the relevant API endpoints from Kenna:
>>
>>- Kenna API Documentation
>>
>>- Kenna Vulnerabilities Export
>>
>>
>> If anyone has experience with similar use cases or can offer any advice,
>> it
>> would be greatly appreciated.
>>
>> Thank you in advance for your help!
>>
>> Best regards
>> Asif Ali
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Abdur-Rahmaan Janhangeer via Python-list]

Idk if you tried Polars, but it seems to work well with JSON data

import polars as pl
pl.read_json("file.json")

Kind Regards,

Abdur-Rahmaan Janhangeer
about  | blog

github 
Mauritius


On Mon, Sep 30, 2024 at 8:00 AM Asif Ali Hirekumbi via Python-list <
python-list@python.org> wrote:

> Dear Python Experts,
>
> I am working with the Kenna Application's API to retrieve vulnerability
> data. The API endpoint provides a single, massive JSON file in gzip format,
> approximately 60 GB in size. Handling such a large dataset in one go is
> proving to be quite challenging, especially in terms of memory management.
>
> I am looking for guidance on how to efficiently stream this data and
> process it in chunks using Python. Specifically, I am wondering if there’s
> a way to use the requests library or any other libraries that would allow
> us to pull data from the API endpoint in a memory-efficient manner.
>
> Here are the relevant API endpoints from Kenna:
>
>- Kenna API Documentation
>
>- Kenna Vulnerabilities Export
>
>
> If anyone has experience with similar use cases or can offer any advice, it
> would be greatly appreciated.
>
> Thank you in advance for your help!
>
> Best regards
> Asif Ali
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Tue, Aug 6, 2024 at 10:53 AM Bill Deegan 
wrote:

> I’m not looking through all the packages you have installed
>
> (Ctrl-F is your friend - - - )


> What version of python is installed on your system?
>
> There are actually 3 versions installed - - - which is why the whole list
was posted.
Your question was really not clear given the context.

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Bill Deegan via Python-list]

I’m not looking through all the packages you have installed

What version of python is installed on your system?


On Tue, Aug 6, 2024 at 5:24 AM o1bigtenor  wrote:

>
>
> On Mon, Aug 5, 2024 at 10:36 PM Bill Deegan 
> wrote:
>
>> why reply to me instead of to the list?
>> It's generally considered bad form to do so.
>>
>
> Got it - - - except this list wants only reply all the next one wants only
> reply and
> keeping straight which is which isn't always happening. I did apologize
> and asked
> you to advise in the next one whether you even wanted to be included
> (which you haven't
> responded to).
>
>>
>>
>> Do you have any python 3 installed on your system?
>> Or python 2.7?
>> If not, can you install such via system package?
>>
>
>
>  # dpkg -l | grep python*
> ii  2to3   3.11.2-1
>   all  2to3 binary using python3
> ii  idle-python3.113.11.2-6+deb12u2
>   all  IDE for Python (v3.11) using Tkinter
> ii  ipython3   8.5.0-4
>  all  Enhanced interactive Python 3 shell
> ii  libboost-python1.74.0  1.74.0+ds1-21
>  amd64Boost.Python Library
> ii  libpython3-all-dev:amd64   3.11.2-1+b1
>  amd64package depending on all supported Python 3
> development packages
> ii  libpython3-dev:amd64   3.11.2-1+b1
>  amd64header files and a static library for Python
> (default)
> ii  libpython3-stdlib:amd643.11.2-1+b1
>  amd64interactive high-level object-oriented language
> (default python3 version)
> rc  libpython3.10-minimal:amd643.10.9-1
>   amd64Minimal subset of the Python language (version
> 3.10)
> ii  libpython3.11:amd643.11.2-6+deb12u2
>   amd64Shared Python runtime library (version 3.11)
> ii  libpython3.11-dev:amd643.11.2-6+deb12u2
>   amd64Header files and a static library for Python
> (v3.11)
> ii  libpython3.11-minimal:amd643.11.2-6+deb12u2
>   amd64Minimal subset of the Python language (version
> 3.11)
> ii  libpython3.11-stdlib:amd64 3.11.2-6+deb12u2
>   amd64Interactive high-level object-oriented language
> (standard library, version 3.11)
> ii  libpython3.11-testsuite3.11.2-6+deb12u2
>   all  Testsuite for the Python standard library (v3.11)
> ii  libpython3.9-minimal:amd64 3.9.13-1
>   amd64Minimal subset of the Python language (version
> 3.9)
> ii  libpython3.9-stdlib:amd64  3.9.13-1
>   amd64Interactive high-level object-oriented language
> (standard library, version 3.9)
> ii  python-apsw-doc3.40.0.0-2
>   all  documentation for python-apsw
> ii  python-apt-common  2.6.0
>  all  Python interface to libapt-pkg (locales)
> ii  python-apt-common-devuan   2.5.3devuan2
>   all  Templates for aptitude
> ii  python-attr-doc22.2.0-1
>   all  documentation for the attrs Python library
> ii  python-babel-localedata2.10.3-1
>   all  tools for internationalizing Python applications
> - locale data files
> ii  python-cryptography-doc38.0.4-3
>   all  Python library exposing cryptographic recipes
> and primitives (documentation)
> ii  python-cycler-doc  0.11.0-1
>   all  composable kwarg iterator (documentation)
> ii  python-gmpy2-common2.1.2-2
>  all  common files for python3-gmpy2
> ii  python-ipython-doc 8.5.0-4
>  all  Enhanced interactive Python shell (documentation)
> ii  python-markdown-doc3.4.1-2
>  all  text-to-HTML conversion library/tool
> (documentation)
> ii  python-matplotlib-data 3.6.3-1
>  all  Python based plotting system (data package)
> ii  python-matplotlib-doc  3.5.2-4
>  all  Python based plotting system (documentation
> package)
> ii  python-mpmath-doc  1.2.1-2
>  all  library for arbitrary-precision floating-point
> arithmetic - Documentation
> ii  python-numpy-doc   1:1.23.5-2
>   all  NumPy documentation
> ii  python-pexpect-doc 4.8.0-4
>  al

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Mon, Aug 5, 2024 at 10:36 PM Bill Deegan 
wrote:

> why reply to me instead of to the list?
> It's generally considered bad form to do so.
>

Got it - - - except this list wants only reply all the next one wants only
reply and
keeping straight which is which isn't always happening. I did apologize and
asked
you to advise in the next one whether you even wanted to be included (which
you haven't
responded to).

>
>
> Do you have any python 3 installed on your system?
> Or python 2.7?
> If not, can you install such via system package?
>


 # dpkg -l | grep python*
ii  2to3   3.11.2-1
all  2to3 binary using python3
ii  idle-python3.113.11.2-6+deb12u2
all  IDE for Python (v3.11) using Tkinter
ii  ipython3   8.5.0-4
 all  Enhanced interactive Python 3 shell
ii  libboost-python1.74.0  1.74.0+ds1-21
 amd64Boost.Python Library
ii  libpython3-all-dev:amd64   3.11.2-1+b1
 amd64package depending on all supported Python 3
development packages
ii  libpython3-dev:amd64   3.11.2-1+b1
 amd64header files and a static library for Python
(default)
ii  libpython3-stdlib:amd643.11.2-1+b1
 amd64interactive high-level object-oriented language
(default python3 version)
rc  libpython3.10-minimal:amd643.10.9-1
amd64Minimal subset of the Python language (version
3.10)
ii  libpython3.11:amd643.11.2-6+deb12u2
amd64Shared Python runtime library (version 3.11)
ii  libpython3.11-dev:amd643.11.2-6+deb12u2
amd64Header files and a static library for Python
(v3.11)
ii  libpython3.11-minimal:amd643.11.2-6+deb12u2
amd64Minimal subset of the Python language (version
3.11)
ii  libpython3.11-stdlib:amd64 3.11.2-6+deb12u2
amd64Interactive high-level object-oriented language
(standard library, version 3.11)
ii  libpython3.11-testsuite3.11.2-6+deb12u2
all  Testsuite for the Python standard library (v3.11)
ii  libpython3.9-minimal:amd64 3.9.13-1
amd64Minimal subset of the Python language (version 3.9)
ii  libpython3.9-stdlib:amd64  3.9.13-1
amd64Interactive high-level object-oriented language
(standard library, version 3.9)
ii  python-apsw-doc3.40.0.0-2
all  documentation for python-apsw
ii  python-apt-common  2.6.0
 all  Python interface to libapt-pkg (locales)
ii  python-apt-common-devuan   2.5.3devuan2
all  Templates for aptitude
ii  python-attr-doc22.2.0-1
all  documentation for the attrs Python library
ii  python-babel-localedata2.10.3-1
all  tools for internationalizing Python applications -
locale data files
ii  python-cryptography-doc38.0.4-3
all  Python library exposing cryptographic recipes and
primitives (documentation)
ii  python-cycler-doc  0.11.0-1
all  composable kwarg iterator (documentation)
ii  python-gmpy2-common2.1.2-2
 all  common files for python3-gmpy2
ii  python-ipython-doc 8.5.0-4
 all  Enhanced interactive Python shell (documentation)
ii  python-markdown-doc3.4.1-2
 all  text-to-HTML conversion library/tool
(documentation)
ii  python-matplotlib-data 3.6.3-1
 all  Python based plotting system (data package)
ii  python-matplotlib-doc  3.5.2-4
 all  Python based plotting system (documentation
package)
ii  python-mpmath-doc  1.2.1-2
 all  library for arbitrary-precision floating-point
arithmetic - Documentation
ii  python-numpy-doc   1:1.23.5-2
all  NumPy documentation
ii  python-pexpect-doc 4.8.0-4
 all  Python module for automating interactive
applications (documentation)
ii  python-pil-doc 9.4.0-1.1+deb12u1
 all  Examples for the Python Imaging Library
ii  python-pygments-doc2.14.0+dfsg-1
 all  documentation for the Pygments
ii  python-scipy-doc

Re: Help needed - - running into issues with python and its tools

[Bill Deegan via Python-list]

why reply to me instead of to the list?
It's generally considered bad form to do so.


Do you have any python 3 installed on your system?
Or python 2.7?
If not, can you install such via system package?

-Bill



On Mon, Aug 5, 2024 at 6:06 PM o1bigtenor  wrote:

>
>
> On Mon, Aug 5, 2024 at 5:28 PM Bill Deegan via Python-list <
> python-list@python.org> wrote:
>
>> Did Mats suggestion of:
>> python3.13 -m venv new_venv
>> $ new_venv/bin/python --version
>> Python 3.13.0b4
>> $ source new_venv/bin/activate
>>
>> Not work?
>> That should work on any system, with any system installl python.
>> It's not trying to modify the system installed python in anyway...
>>
>> If not, please paste the error output you're getting.
>>
>
>  # python3.13 -m venv new_venv
> -bash: python3.13: command not found
>
> $ python3.13 -m venv new_venv
> bash: python3.13: command not found
>
> There you have it - - - the first one run as superuser and the second as
> usr.
>
> Regards
>
>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Bill Deegan via Python-list]

Did Mats suggestion of:
python3.13 -m venv new_venv
$ new_venv/bin/python --version
Python 3.13.0b4
$ source new_venv/bin/activate

Not work?
That should work on any system, with any system installl python.
It's not trying to modify the system installed python in anyway...

If not, please paste the error output you're getting.


On Mon, Aug 5, 2024 at 3:13 PM Mats Wichmann via Python-list <
python-list@python.org> wrote:

> On 8/5/24 15:17, o1bigtenor via Python-list wrote:
>
> >> That's something like
> >>
> >> pyenv install 3.12.4
> >>
> >
> > $ pyenv install 3.12.4
> > bash: pyenv: command not found
> >
>
> > pyenv is not a 'global' package
> >
> > there is a mountain of /root/.pyenv  files though
> > there is also quite a number of /root/.pyenv/plugins/pyenv-virtualenv/
> > files
> >
> > when looking in the /root/.pyenv files I can find options for all the
> older
> > version of python
> > but none for anything newer than what is on my system
> >
> > is there something else to install to achieve this 'version freedom' that
> > pyenv promises?
>
> It has to go somewhere your shell can find it.  Mine is a shell
> function, but it was set up so many years ago I don't remember details.
> It's presumably the pyenv installation instructions...
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Mats Wichmann via Python-list]

On 8/5/24 15:17, o1bigtenor via Python-list wrote:


That's something like

pyenv install 3.12.4



$ pyenv install 3.12.4
bash: pyenv: command not found




pyenv is not a 'global' package

there is a mountain of /root/.pyenv  files though
there is also quite a number of /root/.pyenv/plugins/pyenv-virtualenv/
files

when looking in the /root/.pyenv files I can find options for all the older
version of python
but none for anything newer than what is on my system

is there something else to install to achieve this 'version freedom' that
pyenv promises?


It has to go somewhere your shell can find it.  Mine is a shell 
function, but it was set up so many years ago I don't remember details. 
It's presumably the pyenv installation instructions...




--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Mon, Aug 5, 2024 at 3:56 PM Mats Wichmann  wrote:

>
> > On Mon, Aug 5, 2024 at 8:51 AM Mats Wichmann  > > wrote:
> >
> > On 8/5/24 06:48, o1bigtenor via Python-list wrote:
> >  > On Sun, Aug 4, 2024 at 8:49 AM Mats Wichmann via Python-list <
> >  > python-list@python.org > wrote:
> >  >
> >  >> On 8/3/24 20:03, o1bigtenor via Python-list wrote:
> >  >>
> >  >>> My question was, is and will be (and the doc absolutely doesn't
> > cover it)
> >  >>> how do I install a different version in the venv so that python
> > 3.11.x on
> >  >>> the
> >  >>> system is not discombobulated by the python 3.12.x in the venv.
> >  >>> That python 3.12 would let me run the tools needed.
> >  >>> (Its the how to install the next version of python that I just
> > haven't
> >  >> been
> >  >>> able to find information on - - - and I would be looking for
> > information
> >  >>> on how to install on a *nix.)
> >  >>
> >  >> To get a different Python "in" the venv, you use the version you
> > want in
> >  >> the construction of the venv. For example:
> >  >>
> >  >>
> >  >> $ python3.13 -m venv new_venv
> >  >> $ new_venv/bin/python --version
> >  >> Python 3.13.0b4
> >  >> $ source new_venv/bin/activate
> >  >>
> >  >>
> >  >   "https://peps.python.org/pep-0668/
> >  PEP 668, which prevents pip from
> >  > interacting with the OS installed python. This change has been
> > done in red
> >  > hat and other distros too . . . "
> >  >
> >  > similarly your first command produces and error code for the same
> > reason.
> >  >
> >  > Sorry - - - not my policy - - -
> >
> > What? Yes, the *system* pip should have some restrictions, if it's a
> > system mainly managed by a package manager.
> >
> > Setting up a venv is the *expected* approach to such situations, and
> > creating one doesn't cause any problems. You end up with a pip in the
> > activated venv that's going to install to a different path (the one
> in
> > the venv), and will not be marked as externally managed, as the
> package
> > manager has no control over that path.
> >
> > That's the whole point.  What error are you getting?  The venv
> > module is
> > not the pip module so restrictions on the system pip have nothing to
> do
> > with it.
> >
> > set up pyenv
> > activated a venv
> > trying to install python3.12 into it
> >
> > 1. download of python3.12 (blahblahblahetc).deb will not install
> > 2. download of python3.12.tar.xz similarly will not install
> >
> > (venv2) memyself@devuanbigbox:~$ pip install
> > /home/memyself/Downloads/Python-3.12.4.tar.xz
> > Processing ./Downloads/Python-3.12.4.tar.xz
> > ERROR: file:///home/memyself/Downloads/Python-3.12.4.tar.xz does not
> > appear to be a Python project: neither 'setup.py' nor 'pyproject.toml'
> > found.
> >
> > seems that I need a different version (installable as it were) of
> > python3.12
> > or my approach is all wrong!
>
>
> you can't install Python "into" a venv.
>
> you use a version of Python as the base when *creating* a venv, the venv
> will use the same binary as the base python (symlinks if possible, as in
> the Linux case), but with different paths.
>
> Since you've already got pyenv, use it to build the version you want to
> use - I think you said there wasn't a deb for 3.12 in your distro?
>

correct


> That's something like
>
> pyenv install 3.12.4
>

$ pyenv install 3.12.4
bash: pyenv: command not found


>
> that will use the build recipe it has... and hopefully work.  Distro
> Pythons sometimes have some strange setups that are hard to reproduce.
> Pyenv knows how to build micropython, too, should it ever come to that.
>
> If you indeed found a deb for the right Python, use apt to install it,
> and then use *that* Python to create your venv.
>
> If you have the pyenv-virtualenv plugin, you can ask it to make the
> virtualenv for you, if pyenv built the Python
>
>
pyenv is not a 'global' package

there is a mountain of /root/.pyenv  files though
there is also quite a number of /root/.pyenv/plugins/pyenv-virtualenv/
files

when looking in the /root/.pyenv files I can find options for all the older
version of python
but none for anything newer than what is on my system

is there something else to install to achieve this 'version freedom' that
pyenv promises?

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Mon, Aug 5, 2024 at 3:55 PM Bill Deegan 
wrote:

> Your approach is wrong.
> You don't build python from source using pip.
>
> You don't install new versions of python into a venv either.
>
> Have you read the following?
> https://docs.micropython.org/en/latest/esp8266/tutorial/intro.html
>
> That seems to have instructions for what you want to do..
>

See item 1.4 - - - - that's where I'm at (and that's where the problems
are hidden at as well!)

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Mats Wichmann via Python-list]

On 8/5/24 14:39, o1bigtenor wrote:
Matt - if you would rather that you were not included in the address 
list - -

please advise.

On Mon, Aug 5, 2024 at 8:51 AM Mats Wichmann > wrote:


On 8/5/24 06:48, o1bigtenor via Python-list wrote:
 > On Sun, Aug 4, 2024 at 8:49 AM Mats Wichmann via Python-list <
 > python-list@python.org > wrote:
 >
 >> On 8/3/24 20:03, o1bigtenor via Python-list wrote:
 >>
 >>> My question was, is and will be (and the doc absolutely doesn't
cover it)
 >>> how do I install a different version in the venv so that python
3.11.x on
 >>> the
 >>> system is not discombobulated by the python 3.12.x in the venv.
 >>> That python 3.12 would let me run the tools needed.
 >>> (Its the how to install the next version of python that I just
haven't
 >> been
 >>> able to find information on - - - and I would be looking for
information
 >>> on how to install on a *nix.)
 >>
 >> To get a different Python "in" the venv, you use the version you
want in
 >> the construction of the venv. For example:
 >>
 >>
 >> $ python3.13 -m venv new_venv
 >> $ new_venv/bin/python --version
 >> Python 3.13.0b4
 >> $ source new_venv/bin/activate
 >>
 >>
 >   "https://peps.python.org/pep-0668/
 PEP 668, which prevents pip from
 > interacting with the OS installed python. This change has been
done in red
 > hat and other distros too . . . "
 >
 > similarly your first command produces and error code for the same
reason.
 >
 > Sorry - - - not my policy - - -

What? Yes, the *system* pip should have some restrictions, if it's a
system mainly managed by a package manager.

Setting up a venv is the *expected* approach to such situations, and
creating one doesn't cause any problems. You end up with a pip in the
activated venv that's going to install to a different path (the one in
the venv), and will not be marked as externally managed, as the package
manager has no control over that path.

That's the whole point.  What error are you getting?  The venv
module is
not the pip module so restrictions on the system pip have nothing to do
with it.

set up pyenv
activated a venv
trying to install python3.12 into it

1. download of python3.12 (blahblahblahetc).deb will not install
2. download of python3.12.tar.xz similarly will not install

(venv2) memyself@devuanbigbox:~$ pip install 
/home/memyself/Downloads/Python-3.12.4.tar.xz

Processing ./Downloads/Python-3.12.4.tar.xz
ERROR: file:///home/memyself/Downloads/Python-3.12.4.tar.xz does not 
appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' 
found.


seems that I need a different version (installable as it were) of 
python3.12

or my approach is all wrong!



you can't install Python "into" a venv.

you use a version of Python as the base when *creating* a venv, the venv 
will use the same binary as the base python (symlinks if possible, as in 
the Linux case), but with different paths.


Since you've already got pyenv, use it to build the version you want to 
use - I think you said there wasn't a deb for 3.12 in your distro? 
That's something like


pyenv install 3.12.4

that will use the build recipe it has... and hopefully work.  Distro 
Pythons sometimes have some strange setups that are hard to reproduce. 
Pyenv knows how to build micropython, too, should it ever come to that.


If you indeed found a deb for the right Python, use apt to install it, 
and then use *that* Python to create your venv.


If you have the pyenv-virtualenv plugin, you can ask it to make the 
virtualenv for you, if pyenv built the Python





--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Bill Deegan via Python-list]

Your approach is wrong.
You don't build python from source using pip.

You don't install new versions of python into a venv either.

Have you read the following?
https://docs.micropython.org/en/latest/esp8266/tutorial/intro.html

That seems to have instructions for what you want to do..

-Bill



On Mon, Aug 5, 2024 at 1:41 PM o1bigtenor via Python-list <
python-list@python.org> wrote:

> Matt - if you would rather that you were not included in the address list -
> -
> please advise.
>
> On Mon, Aug 5, 2024 at 8:51 AM Mats Wichmann  wrote:
>
> > On 8/5/24 06:48, o1bigtenor via Python-list wrote:
> > > On Sun, Aug 4, 2024 at 8:49 AM Mats Wichmann via Python-list <
> > > python-list@python.org> wrote:
> > >
> > >> On 8/3/24 20:03, o1bigtenor via Python-list wrote:
> > >>
> > >>> My question was, is and will be (and the doc absolutely doesn't cover
> > it)
> > >>> how do I install a different version in the venv so that python
> 3.11.x
> > on
> > >>> the
> > >>> system is not discombobulated by the python 3.12.x in the venv.
> > >>> That python 3.12 would let me run the tools needed.
> > >>> (Its the how to install the next version of python that I just
> haven't
> > >> been
> > >>> able to find information on - - - and I would be looking for
> > information
> > >>> on how to install on a *nix.)
> > >>
> > >> To get a different Python "in" the venv, you use the version you want
> in
> > >> the construction of the venv. For example:
> > >>
> > >>
> > >> $ python3.13 -m venv new_venv
> > >> $ new_venv/bin/python --version
> > >> Python 3.13.0b4
> > >> $ source new_venv/bin/activate
> > >>
> > >>
> > >   "https://peps.python.org/pep-0668/ PEP 668, which prevents pip from
> > > interacting with the OS installed python. This change has been done in
> > red
> > > hat and other distros too . . . "
> > >
> > > similarly your first command produces and error code for the same
> reason.
> > >
> > > Sorry - - - not my policy - - -
> >
> > What? Yes, the *system* pip should have some restrictions, if it's a
> > system mainly managed by a package manager.
> >
> > Setting up a venv is the *expected* approach to such situations, and
> > creating one doesn't cause any problems. You end up with a pip in the
> > activated venv that's going to install to a different path (the one in
> > the venv), and will not be marked as externally managed, as the package
> > manager has no control over that path.
> >
> > That's the whole point.  What error are you getting?  The venv module is
> > not the pip module so restrictions on the system pip have nothing to do
> > with it.
> >
> > set up pyenv
> activated a venv
> trying to install python3.12 into it
>
> 1. download of python3.12 (blahblahblahetc).deb will not install
> 2. download of python3.12.tar.xz similarly will not install
>
> (venv2) memyself@devuanbigbox:~$ pip install
> /home/memyself/Downloads/Python-3.12.4.tar.xz
> Processing ./Downloads/Python-3.12.4.tar.xz
> ERROR: file:///home/memyself/Downloads/Python-3.12.4.tar.xz does not appear
> to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.
>
> seems that I need a different version (installable as it were) of
> python3.12
> or my approach is all wrong!
>
> Please advise
>
> TIA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

Matt - if you would rather that you were not included in the address list -
-
please advise.

On Mon, Aug 5, 2024 at 8:51 AM Mats Wichmann  wrote:

> On 8/5/24 06:48, o1bigtenor via Python-list wrote:
> > On Sun, Aug 4, 2024 at 8:49 AM Mats Wichmann via Python-list <
> > python-list@python.org> wrote:
> >
> >> On 8/3/24 20:03, o1bigtenor via Python-list wrote:
> >>
> >>> My question was, is and will be (and the doc absolutely doesn't cover
> it)
> >>> how do I install a different version in the venv so that python 3.11.x
> on
> >>> the
> >>> system is not discombobulated by the python 3.12.x in the venv.
> >>> That python 3.12 would let me run the tools needed.
> >>> (Its the how to install the next version of python that I just haven't
> >> been
> >>> able to find information on - - - and I would be looking for
> information
> >>> on how to install on a *nix.)
> >>
> >> To get a different Python "in" the venv, you use the version you want in
> >> the construction of the venv. For example:
> >>
> >>
> >> $ python3.13 -m venv new_venv
> >> $ new_venv/bin/python --version
> >> Python 3.13.0b4
> >> $ source new_venv/bin/activate
> >>
> >>
> >   "https://peps.python.org/pep-0668/ PEP 668, which prevents pip from
> > interacting with the OS installed python. This change has been done in
> red
> > hat and other distros too . . . "
> >
> > similarly your first command produces and error code for the same reason.
> >
> > Sorry - - - not my policy - - -
>
> What? Yes, the *system* pip should have some restrictions, if it's a
> system mainly managed by a package manager.
>
> Setting up a venv is the *expected* approach to such situations, and
> creating one doesn't cause any problems. You end up with a pip in the
> activated venv that's going to install to a different path (the one in
> the venv), and will not be marked as externally managed, as the package
> manager has no control over that path.
>
> That's the whole point.  What error are you getting?  The venv module is
> not the pip module so restrictions on the system pip have nothing to do
> with it.
>
> set up pyenv
activated a venv
trying to install python3.12 into it

1. download of python3.12 (blahblahblahetc).deb will not install
2. download of python3.12.tar.xz similarly will not install

(venv2) memyself@devuanbigbox:~$ pip install
/home/memyself/Downloads/Python-3.12.4.tar.xz
Processing ./Downloads/Python-3.12.4.tar.xz
ERROR: file:///home/memyself/Downloads/Python-3.12.4.tar.xz does not appear
to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

seems that I need a different version (installable as it were) of
python3.12
or my approach is all wrong!

Please advise

TIA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Mats Wichmann via Python-list]

On 8/5/24 06:48, o1bigtenor via Python-list wrote:

On Sun, Aug 4, 2024 at 8:49 AM Mats Wichmann via Python-list <
python-list@python.org> wrote:


On 8/3/24 20:03, o1bigtenor via Python-list wrote:


My question was, is and will be (and the doc absolutely doesn't cover it)
how do I install a different version in the venv so that python 3.11.x on
the
system is not discombobulated by the python 3.12.x in the venv.
That python 3.12 would let me run the tools needed.
(Its the how to install the next version of python that I just haven't

been

able to find information on - - - and I would be looking for information
on how to install on a *nix.)


To get a different Python "in" the venv, you use the version you want in
the construction of the venv. For example:


$ python3.13 -m venv new_venv
$ new_venv/bin/python --version
Python 3.13.0b4
$ source new_venv/bin/activate



  "https://peps.python.org/pep-0668/ PEP 668, which prevents pip from
interacting with the OS installed python. This change has been done in red
hat and other distros too . . . "

similarly your first command produces and error code for the same reason.

Sorry - - - not my policy - - -


What? Yes, the *system* pip should have some restrictions, if it's a 
system mainly managed by a package manager.


Setting up a venv is the *expected* approach to such situations, and 
creating one doesn't cause any problems. You end up with a pip in the 
activated venv that's going to install to a different path (the one in 
the venv), and will not be marked as externally managed, as the package 
manager has no control over that path.


That's the whole point.  What error are you getting?  The venv module is 
not the pip module so restrictions on the system pip have nothing to do 
with it.




--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Sun, Aug 4, 2024 at 8:49 AM Mats Wichmann via Python-list <
python-list@python.org> wrote:

> On 8/3/24 20:03, o1bigtenor via Python-list wrote:
>
> > My question was, is and will be (and the doc absolutely doesn't cover it)
> > how do I install a different version in the venv so that python 3.11.x on
> > the
> > system is not discombobulated by the python 3.12.x in the venv.
> > That python 3.12 would let me run the tools needed.
> > (Its the how to install the next version of python that I just haven't
> been
> > able to find information on - - - and I would be looking for information
> > on how to install on a *nix.)
>
> To get a different Python "in" the venv, you use the version you want in
> the construction of the venv. For example:
>
>
> $ python3.13 -m venv new_venv
> $ new_venv/bin/python --version
> Python 3.13.0b4
> $ source new_venv/bin/activate
>
>
 "https://peps.python.org/pep-0668/ PEP 668, which prevents pip from
interacting with the OS installed python. This change has been done in red
hat and other distros too . . . "

similarly your first command produces and error code for the same reason.

Sorry - - - not my policy - - -

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Sun, Aug 4, 2024 at 4:24 AM Peter J. Holzer via Python-list <
python-list@python.org> wrote:

> On 2024-08-03 15:17:11 -0500, o1bigtenor via Python-list wrote:
> > One of the tools I need to be able to use is esptools - -  well in the
> > devuan world you need to run that on either Devaun 3 or 5 - - - its just
> > not available on devuan 4.
>
> Couldn't you just upgrade to Devuan 5, then?
>

I like uptime and not measured in hours either so I'm presently not running
Devuan 5
or testing - - - although I used to. Just found that I didn't need the most
recent versions
of most of the tools I was using so find the stability (except for the
bloody browsers)
in stable version is to be appreciated.

>
>
> > Tried installing all the tools I need using downloads and .deb installs
> but
> > then I need to have python3.12 and that's also not part of Devuan4.
>
> It seems weird that something would work with the (presumably) older
> version of Python in Devuan 3 and the (presumably) newer version of
> Python in Devuan 5, but not with the version in Devuan 4.
>

Well - - - pstool was available in a version that worked in Devuan 3,
psytool is available in a version that works in Devuan 4 - - - beyond my
pay
grade as to why there isn't a version available for Devuan 4.

>
>
> > Not versed enough to set up a good venv (if that's possible) so that I
> > could work in that specific venv and have my cake (and get to eat it too
> > (grin!).
>
> You need to install Python first to create a venv. AFAIK there is no way
> to set up a venv first and then install Python into it.
>
> Does Devuan have a testing or unstable suite? You might be able to
> install a newer Python version from that. If not your best bet is to
> install Python from source.
>
>
First question answered earlier.
If I installed python 3.12 from source I would create a hung system for
myself.
(Been there and done that - - - got the T-shirt and it still stinks!)

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Mats Wichmann via Python-list]

On 8/3/24 20:03, o1bigtenor via Python-list wrote:


My question was, is and will be (and the doc absolutely doesn't cover it)
how do I install a different version in the venv so that python 3.11.x on
the
system is not discombobulated by the python 3.12.x in the venv.
That python 3.12 would let me run the tools needed.
(Its the how to install the next version of python that I just haven't been
able to find information on - - - and I would be looking for information
on how to install on a *nix.)


To get a different Python "in" the venv, you use the version you want in 
the construction of the venv. For example:



$ python3.13 -m venv new_venv
$ new_venv/bin/python --version
Python 3.13.0b4
$ source new_venv/bin/activate
...




--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Peter J. Holzer via Python-list]

On 2024-08-03 15:17:11 -0500, o1bigtenor via Python-list wrote:
> One of the tools I need to be able to use is esptools - -  well in the
> devuan world you need to run that on either Devaun 3 or 5 - - - its just
> not available on devuan 4.

Couldn't you just upgrade to Devuan 5, then?


> Tried installing all the tools I need using downloads and .deb installs but
> then I need to have python3.12 and that's also not part of Devuan4.

It seems weird that something would work with the (presumably) older
version of Python in Devuan 3 and the (presumably) newer version of
Python in Devuan 5, but not with the version in Devuan 4.


> Not versed enough to set up a good venv (if that's possible) so that I
> could work in that specific venv and have my cake (and get to eat it too
> (grin!).

You need to install Python first to create a venv. AFAIK there is no way
to set up a venv first and then install Python into it.

Does Devuan have a testing or unstable suite? You might be able to
install a newer Python version from that. If not your best bet is to
install Python from source.

hp


-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Sat, Aug 3, 2024 at 7:11 PM dn via Python-list 
wrote:

> On 4/08/24 09:34, o1bigtenor via Python-list wrote:
> > On Sat, Aug 3, 2024 at 4:06 PM dn via Python-list <
> python-list@python.org>
> > wrote:
> >
> >> On 4/08/24 08:17, o1bigtenor via Python-list wrote:
> >>> Greetings
> >>>
> >>> Looking at ESP8266 and wanting to program it using micropython (really
> >>> don't want to have to learn C++ (not enough hours in the day as it
> >> is!!)).
> >>>
> >>> One of the tools I need to be able to use is esptools - -  well in the
> >>> devuan world you need to run that on either Devaun 3 or 5 - - - its
> just
> >>> not available on devuan 4.
> >>>
> >>> Tried installing all the tools I need using downloads and .deb installs
> >> but
> >>> then I need to have python3.12 and that's also not part of Devuan4.
> >>>
> >>> Not versed enough to set up a good venv (if that's possible) so that I
> >>> could work in that specific venv and have my cake (and get to eat it
> too
> >>> (grin!).
> >>>
> >>> Suggestions - - - ideas - - - please?
> >>
> >> Sorry if this offends, but this is a list of short-cuts and reasons why
> >> they don't work (immediately).
> >>
> >>
> >> Have just come from a discussion about 'how to start a project'. Amongst
> >> the questions to ask are: "what resources do we have (or can add) to
> >> achieve?".
> >>
> >
> > Fair question details interleaved - - -
> >
> >>
> >> In this case, if Python-skill is a "personnel-resource" (and C++ a
> >> "constraint"), will question the ESP over Raspberry Pi (say)?
> >>
> >
> > Well - - - RPi world technical specs is usable from -20 to 60 C (iirc on
> > the top number)
> > and for my project I absolutely need to have usability to at least -40 -
> -
> > could possibly do a bit less but -35 C is a hard requirement so the RPi
> > and Pico (which I would like to use) is out but ESP8266 runs in that -40
> to
> > 65 C
> > range.
>
> Interesting, but creates a mis-match of tools - battles for you to fight...
>

Not really - - - it takes a computer with psytool and about 5 or 6 other
python3.12
programs to load the microcontroller.
It is in writing the program for the microcontroller that micropython is
used not at
all in the loading.

>
>
> >> Why talking of Python 3.12 when the solution involves MicroPython?
> >>
> >
> > Because one uses psytool on one computer to transfer a program to the
> > MicroPython system.  In fact there are a set of tools that need
> Python3.12
> > to be able to do this and therefore the question.
> >
> >>
> >>
> >> Perhaps need to take a step back and look at 'options' - relate needs to
> >> resources, and evaluate the impact of each decision on later ones - as
> >> well as against your personal skills (modify objectives to limits, or
> >> accept that some learning/training will be necessary as pre-requisite to
> >> (being able to) attack the project).
> >>
> >> I have been investigating using a venv but am not finding clear
> directions
> > so
> > that I could set up Python3.12 inside (along with the other needed
> tools).
> > The
> > more I'm looking the less useful most of the information I'm finding is
> > becoming.
> > Therefore I thought I would go to the python gurus for information - - -
> > which I
> > have.
> >
> > So please - - - how do I set up a venv so that I can install and run
> python
> > 3.12
> > (and other needed programs related to 3.12) inside?
>
> If you mean venv itself, which "directions" have you reviewed?
> This one (https://python.land/virtual-environments/virtualenv) seems
> very straight-forward and shows "What's inside a venv?" to include
> python.exe. Given that venv is more-or-less the official/traditional
> solution, what are you doing differently - perhaps the question is
> lacking detail.
>
> Interesting - - - that's the doc I have been reading.

My question was, is and will be (and the doc absolutely doesn't cover it)
how do I install a different version in the venv so that python 3.11.x on
the
system is not discombobulated by the python 3.12.x in the venv.
That python 3.12 would let me run the tools needed.
(Its the how to install the next version of python that I just haven't been
able to find information on - - - and I would be looking for information
on how to install on a *nix.)


> Personally, I'm using Poetry (https://python-poetry.org) which seemed
> just as easy to pick-up; plus pyenv to maintain multiple versions of
> Python on one machine.
>
>
Will give python-poetry a look.

Have been looking at pyenv but that seems to be a whole rat's nest of other
stuff to install and its using a bunch of different tools to get there - -
- is it
necessary - - - yes or no (in the running of multiple python versions on
the
same machine).

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Sat, Aug 3, 2024 at 6:20 PM Cameron Simpson via Python-list <
python-list@python.org> wrote:

> On 03Aug2024 16:34, o1bigtenor  wrote:
> >So please - - - how do I set up a venv so that I can install and run
> >python
> >3.12
> >(and other needed programs related to 3.12) inside?
>
> Maybe this github comment will help with this:
>
> https://github.com/orgs/micropython/discussions/10255#discussioncomment-671
>

Not really.

A computer that has psytool + about 5 or 6 other python 3.12 tools is
needed to load
the microcontroller so I don't see how having venv in the microcontroller
would help the
loading of software onto the microcontroller.

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[dn via Python-list]

On 4/08/24 09:34, o1bigtenor via Python-list wrote:

On Sat, Aug 3, 2024 at 4:06 PM dn via Python-list 
wrote:


On 4/08/24 08:17, o1bigtenor via Python-list wrote:

Greetings

Looking at ESP8266 and wanting to program it using micropython (really
don't want to have to learn C++ (not enough hours in the day as it

is!!)).


One of the tools I need to be able to use is esptools - -  well in the
devuan world you need to run that on either Devaun 3 or 5 - - - its just
not available on devuan 4.

Tried installing all the tools I need using downloads and .deb installs

but

then I need to have python3.12 and that's also not part of Devuan4.

Not versed enough to set up a good venv (if that's possible) so that I
could work in that specific venv and have my cake (and get to eat it too
(grin!).

Suggestions - - - ideas - - - please?


Sorry if this offends, but this is a list of short-cuts and reasons why
they don't work (immediately).


Have just come from a discussion about 'how to start a project'. Amongst
the questions to ask are: "what resources do we have (or can add) to
achieve?".



Fair question details interleaved - - -



In this case, if Python-skill is a "personnel-resource" (and C++ a
"constraint"), will question the ESP over Raspberry Pi (say)?



Well - - - RPi world technical specs is usable from -20 to 60 C (iirc on
the top number)
and for my project I absolutely need to have usability to at least -40 - -
could possibly do a bit less but -35 C is a hard requirement so the RPi
and Pico (which I would like to use) is out but ESP8266 runs in that -40 to
65 C
range.


Interesting, but creates a mis-match of tools - battles for you to fight...



Why talking of Python 3.12 when the solution involves MicroPython?



Because one uses psytool on one computer to transfer a program to the
MicroPython system.  In fact there are a set of tools that need Python3.12
to be able to do this and therefore the question.




Perhaps need to take a step back and look at 'options' - relate needs to
resources, and evaluate the impact of each decision on later ones - as
well as against your personal skills (modify objectives to limits, or
accept that some learning/training will be necessary as pre-requisite to
(being able to) attack the project).

I have been investigating using a venv but am not finding clear directions

so
that I could set up Python3.12 inside (along with the other needed tools).
The
more I'm looking the less useful most of the information I'm finding is
becoming.
Therefore I thought I would go to the python gurus for information - - -
which I
have.

So please - - - how do I set up a venv so that I can install and run python
3.12
(and other needed programs related to 3.12) inside?


If you mean venv itself, which "directions" have you reviewed?
This one (https://python.land/virtual-environments/virtualenv) seems 
very straight-forward and shows "What's inside a venv?" to include 
python.exe. Given that venv is more-or-less the official/traditional 
solution, what are you doing differently - perhaps the question is 
lacking detail.

(see also @Cameron's take)

Personally, I'm using Poetry (https://python-poetry.org) which seemed 
just as easy to pick-up; plus pyenv to maintain multiple versions of 
Python on one machine.


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Cameron Simpson via Python-list]

On 03Aug2024 16:34, o1bigtenor  wrote:
So please - - - how do I set up a venv so that I can install and run 
python

3.12
(and other needed programs related to 3.12) inside?


Maybe this github comment will help with this:
https://github.com/orgs/micropython/discussions/10255#discussioncomment-671
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[o1bigtenor via Python-list]

On Sat, Aug 3, 2024 at 4:06 PM dn via Python-list 
wrote:

> On 4/08/24 08:17, o1bigtenor via Python-list wrote:
> > Greetings
> >
> > Looking at ESP8266 and wanting to program it using micropython (really
> > don't want to have to learn C++ (not enough hours in the day as it
> is!!)).
> >
> > One of the tools I need to be able to use is esptools - -  well in the
> > devuan world you need to run that on either Devaun 3 or 5 - - - its just
> > not available on devuan 4.
> >
> > Tried installing all the tools I need using downloads and .deb installs
> but
> > then I need to have python3.12 and that's also not part of Devuan4.
> >
> > Not versed enough to set up a good venv (if that's possible) so that I
> > could work in that specific venv and have my cake (and get to eat it too
> > (grin!).
> >
> > Suggestions - - - ideas - - - please?
>
> Sorry if this offends, but this is a list of short-cuts and reasons why
> they don't work (immediately).
>
>
> Have just come from a discussion about 'how to start a project'. Amongst
> the questions to ask are: "what resources do we have (or can add) to
> achieve?".
>

Fair question details interleaved - - -

>
> In this case, if Python-skill is a "personnel-resource" (and C++ a
> "constraint"), will question the ESP over Raspberry Pi (say)?
>

Well - - - RPi world technical specs is usable from -20 to 60 C (iirc on
the top number)
and for my project I absolutely need to have usability to at least -40 - -
could possibly do a bit less but -35 C is a hard requirement so the RPi
and Pico (which I would like to use) is out but ESP8266 runs in that -40 to
65 C
range.

>
> Why talking of Python 3.12 when the solution involves MicroPython?
>

Because one uses psytool on one computer to transfer a program to the
MicroPython system.  In fact there are a set of tools that need Python3.12
to be able to do this and therefore the question.

>
>
> Perhaps need to take a step back and look at 'options' - relate needs to
> resources, and evaluate the impact of each decision on later ones - as
> well as against your personal skills (modify objectives to limits, or
> accept that some learning/training will be necessary as pre-requisite to
> (being able to) attack the project).
>
> I have been investigating using a venv but am not finding clear directions
so
that I could set up Python3.12 inside (along with the other needed tools).
The
more I'm looking the less useful most of the information I'm finding is
becoming.
Therefore I thought I would go to the python gurus for information - - -
which I
have.

So please - - - how do I set up a venv so that I can install and run python
3.12
(and other needed programs related to 3.12) inside?

TIA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[dn via Python-list]

On 4/08/24 08:17, o1bigtenor via Python-list wrote:

Greetings

Looking at ESP8266 and wanting to program it using micropython (really
don't want to have to learn C++ (not enough hours in the day as it is!!)).

One of the tools I need to be able to use is esptools - -  well in the
devuan world you need to run that on either Devaun 3 or 5 - - - its just
not available on devuan 4.

Tried installing all the tools I need using downloads and .deb installs but
then I need to have python3.12 and that's also not part of Devuan4.

Not versed enough to set up a good venv (if that's possible) so that I
could work in that specific venv and have my cake (and get to eat it too
(grin!).

Suggestions - - - ideas - - - please?


Sorry if this offends, but this is a list of short-cuts and reasons why 
they don't work (immediately).



Have just come from a discussion about 'how to start a project'. Amongst 
the questions to ask are: "what resources do we have (or can add) to 
achieve?".


In this case, if Python-skill is a "personnel-resource" (and C++ a 
"constraint"), will question the ESP over Raspberry Pi (say)?


Why talking of Python 3.12 when the solution involves MicroPython?


Perhaps need to take a step back and look at 'options' - relate needs to 
resources, and evaluate the impact of each decision on later ones - as 
well as against your personal skills (modify objectives to limits, or 
accept that some learning/training will be necessary as pre-requisite to 
(being able to) attack the project).


--
Regards =dn

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: help

[Thomas Passin via Python-list]

On 5/12/2024 7:56 PM, Enrder via Python-list wrote:

good tader

I need help to install the bcml, and is that after installing the python
and I go to the command prompt and put ''pip install bcml'' to install it
tells me "pip" is not recognized as an internal or external command,
program or executable batch file.

I have tried everything, uninstalling and installing, restoring it,
activating and deactivating all the checkboxes that appear in the options
section, I searched the internet and the same thing keeps happening to me.



If you can help me, I would appreciate it very much.


Launch pip using the following command.  If you don't run python using 
the name "python" then use the name you usually use (e.g., "py", 
"python3", etc.) -


python -m pip install bcml

This command causes Python to load and run its own copy of pip.  So the 
fact that the operating system can't find it doesn't matter.  You should 
always run pip this way, especially if you end up with several different 
versions of Python on the same computer.


--
https://mail.python.org/mailman/listinfo/python-list

Re: help: pandas and 2d table

[jak via Python-list]

Stefan Ram ha scritto:

jak  wrote or quoted:

Stefan Ram ha scritto:

df = df.where( df == 'zz' ).stack().reset_index()
result ={ 'zz': list( zip( df.iloc[ :, 0 ], df.iloc[ :, 1 ]))}

Since I don't know Pandas, I will need a month at least to understand
these 2 lines of code. Thanks again.


   Here's a technique to better understand such code:

   Transform it into a program with small statements and small
   expressions with no more than one call per statement if possible.
   (After each litte change check that the output stays the same.)

import pandas as pd

# Warning! Will overwrite the file 'file_20240412201813_tmp_DML.csv'!
with open( 'file_20240412201813_tmp_DML.csv', 'w' )as out:
 print( '''obj,foo1,foo2,foo3,foo4,foo5,foo6
foo1,aa,ab,zz,ad,ae,af
foo2,ba,bb,bc,bd,zz,bf
foo3,ca,zz,cc,cd,ce,zz
foo4,da,db,dc,dd,de,df
foo5,ea,eb,ec,zz,ee,ef
foo6,fa,fb,fc,fd,fe,ff''', file=out )
# Note the "index_col=0" below, which is important here!
df = pd.read_csv( 'file_20240412201813_tmp_DML.csv', index_col=0 )

selection = df.where( df == 'zz' )
selection_stack = selection.stack()
df = selection_stack.reset_index()
df0 = df.iloc[ :, 0 ]
df1 = df.iloc[ :, 1 ]
z = zip( df0, df1 )
l = list( z )
result ={ 'zz': l }
print( result )

   I suggest to next insert print statements to print each intermediate
   value:

# Note the "index_col=0" below, which is important here!
df = pd.read_csv( 'file_20240412201813_tmp_DML.csv', index_col=0 )
print( 'df = \n', type( df ), ':\n"', df, '"\n' )

selection = df.where( df == 'zz' )
print( "result of where( df == 'zz' ) = \n", type( selection ), ':\n"',
   selection, '"\n' )

selection_stack = selection.stack()
print( 'result of stack() = \n', type( selection_stack ), ':\n"',
   selection_stack, '"\n' )

df = selection_stack.reset_index()
print( 'result of reset_index() = \n', type( df ), ':\n"', df, '"\n' )

df0 = df.iloc[ :, 0 ]
print( 'value of .iloc[ :, 0 ]= \n', type( df0 ), ':\n"', df0, '"\n' )

df1 = df.iloc[ :, 1 ]
print( 'value of .iloc[ :, 1 ] = \n', type( df1 ), ':\n"', df1, '"\n' )

z = zip( df0, df1 )
print( 'result of zip( df0, df1 )= \n', type( z ), ':\n"', z, '"\n' )

l = list( z )
print( 'result of list( z )= \n', type( l ), ':\n"', l, '"\n' )

result ={ 'zz': l }
print( "value of { 'zz': l }= \n", type( result ), ':\n"',
   result, '"\n' )

print( result )

   Now you can see what each single step does!

df =
   :
"  foo1 foo2 foo3 foo4 foo5 foo6
obj
foo1   aa   ab   zz   ad   ae   af
foo2   ba   bb   bc   bd   zz   bf
foo3   ca   zz   cc   cd   ce   zz
foo4   da   db   dc   dd   de   df
foo5   ea   eb   ec   zz   ee   ef
foo6   fa   fb   fc   fd   fe   ff "

result of where( df == 'zz' ) =
   :
"  foo1 foo2 foo3 foo4 foo5 foo6
obj
foo1  NaN  NaN   zz  NaN  NaN  NaN
foo2  NaN  NaN  NaN  NaN   zz  NaN
foo3  NaN   zz  NaN  NaN  NaN   zz
foo4  NaN  NaN  NaN  NaN  NaN  NaN
foo5  NaN  NaN  NaN   zz  NaN  NaN
foo6  NaN  NaN  NaN  NaN  NaN  NaN "

result of stack() =
   :
" obj
foo1  foo3zz
foo2  foo5zz
foo3  foo2zz
   foo6zz
foo5  foo4zz
dtype: object "

result of reset_index() =
   :
" obj level_1   0
0  foo1foo3  zz
1  foo2foo5  zz
2  foo3foo2  zz
3  foo3foo6  zz
4  foo5foo4  zz "

value of .iloc[ :, 0 ]=
   :
" 0foo1
1foo2
2foo3
3foo3
4foo5
Name: obj, dtype: object "

value of .iloc[ :, 1 ] =
   :
" 0foo3
1foo5
2foo2
3foo6
4foo4
Name: level_1, dtype: object "

result of zip( df0, df1 )=
   :
" "

result of list( z )=
   :
" [('foo1', 'foo3'), ('foo2', 'foo5'), ('foo3', 'foo2'), ('foo3', 'foo6'), ('foo5', 
'foo4')]"

value of { 'zz': l }=
   :
" {'zz': [('foo1', 'foo3'), ('foo2', 'foo5'), ('foo3', 'foo2'), ('foo3', 'foo6'), 
('foo5', 'foo4')]}"

{'zz': [('foo1', 'foo3'), ('foo2', 'foo5'), ('foo3', 'foo2'), ('foo3', 'foo6'), 
('foo5', 'foo4')]}

   The script reads a CSV file and stores the data in a Pandas
   DataFrame object named "df". The "index_col=0" parameter tells
   Pandas to use the first column as the index for the DataFrame,
   which is kinda like column headers.

   The "where" creates a new DataFrame selection that contains
   the same data as df, but with all values replaced by NaN (Not
   a Number) except for the values that are equal to 'zz'.

   "stack" returns a Series with a multi-level index created
   by pivoting the columns. Here it gives a Series with the
   row-col-addresses of a all the non-NaN values. The general
   meaning of "stack" might be the most complex operation of
   this script. It's explained in the pandas manual (see there).

   "reset_index" then just transforms this Series back into a
   DataFrame, and ".iloc[ :, 0 ]" and ".iloc[ :, 1 ]" are the
   first and second column, respectively, of that DataFrame. These
   then are zipped to get the desired form as a list of pairs.



And this is a technique very similar to reverse engineering. Thanks for
the explanation and examples. All this is really clear and I was able to
follow i

Re: help: pandas and 2d table

[jak via Python-list]

Stefan Ram ha scritto:

df = df.where( df == 'zz' ).stack().reset_index()
result ={ 'zz': list( zip( df.iloc[ :, 0 ], df.iloc[ :, 1 ]))}


Since I don't know Pandas, I will need a month at least to understand
these 2 lines of code. Thanks again.
--
https://mail.python.org/mailman/listinfo/python-list

Re: help: pandas and 2d table

[Tim Williams via Python-list]

On Sat, Apr 13, 2024 at 1:10 PM Mats Wichmann via Python-list <
python-list@python.org> wrote:

> On 4/13/24 07:00, jak via Python-list wrote:
>
> doesn't Pandas have a "where" method that can do this kind of thing? Or
> doesn't it match what you are looking for?  Pretty sure numpy does, but
> that's a lot to bring in if you don't need the rest of numpy.
>
> pandas.DataFrame.where — pandas 2.2.2 documentation (pydata.org)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: help: pandas and 2d table

[Mats Wichmann via Python-list]

On 4/13/24 07:00, jak via Python-list wrote:

Stefan Ram ha scritto:

jak  wrote or quoted:

Would you show me the path, please?


   I was not able to read xls here, so I used csv instead; Warning:
   the script will overwrite file "file_20240412201813_tmp_DML.csv"!

import pandas as pd

with open( 'file_20240412201813_tmp_DML.csv', 'w' )as out:
 print( '''obj,foo1,foo2,foo3,foo4,foo5,foo6
foo1,aa,ab,zz,ad,ae,af
foo2,ba,bb,bc,bd,zz,bf
foo3,ca,zz,cc,cd,ce,zz
foo4,da,db,dc,dd,de,df
foo5,ea,eb,ec,zz,ee,ef
foo6,fa,fb,fc,fd,fe,ff''', file=out )

df = pd.read_csv( 'file_20240412201813_tmp_DML.csv' )

result = {}

for rownum, row in df.iterrows():
 iterator = row.items()
 _, rowname = next( iterator )
 for colname, value in iterator:
 if value not in result: result[ value ]= []
 result[ value ].append( ( rowname, colname ))

print( result )



In reality what I wanted to achieve was this:

     what = 'zz'
     result = {what: []}

     for rownum, row in df.iterrows():
     iterator = row.items()
     _, rowname = next(iterator)
     for colname, value in iterator:
     if value == what:
     result[what] += [(rowname, colname)]
     print(result)

In any case, thank you again for pointing me in the right direction. I
had lost myself looking for a pandas method that would do this in a
single shot or almost.




doesn't Pandas have a "where" method that can do this kind of thing? Or 
doesn't it match what you are looking for?  Pretty sure numpy does, but 
that's a lot to bring in if you don't need the rest of numpy.



--
https://mail.python.org/mailman/listinfo/python-list

Re: help: pandas and 2d table

[jak via Python-list]

Stefan Ram ha scritto:

jak  wrote or quoted:

Would you show me the path, please?


   I was not able to read xls here, so I used csv instead; Warning:
   the script will overwrite file "file_20240412201813_tmp_DML.csv"!

import pandas as pd

with open( 'file_20240412201813_tmp_DML.csv', 'w' )as out:
 print( '''obj,foo1,foo2,foo3,foo4,foo5,foo6
foo1,aa,ab,zz,ad,ae,af
foo2,ba,bb,bc,bd,zz,bf
foo3,ca,zz,cc,cd,ce,zz
foo4,da,db,dc,dd,de,df
foo5,ea,eb,ec,zz,ee,ef
foo6,fa,fb,fc,fd,fe,ff''', file=out )

df = pd.read_csv( 'file_20240412201813_tmp_DML.csv' )

result = {}

for rownum, row in df.iterrows():
 iterator = row.items()
 _, rowname = next( iterator )
 for colname, value in iterator:
 if value not in result: result[ value ]= []
 result[ value ].append( ( rowname, colname ))

print( result )



In reality what I wanted to achieve was this:

what = 'zz'
result = {what: []}

for rownum, row in df.iterrows():
iterator = row.items()
_, rowname = next(iterator)
for colname, value in iterator:
if value == what:
result[what] += [(rowname, colname)]
print(result)

In any case, thank you again for pointing me in the right direction. I
had lost myself looking for a pandas method that would do this in a
single shot or almost.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Help Needed With a Python Gaming Module

[Thomas Passin via Python-list]

On 4/3/2024 3:06 PM, WordWeaver Evangelist via Python-list wrote:

Hello everyone! It has been a l-o-n-g time -- nine years in fact --since I last 
participated on this mailing list.


[snip]

3. You are very familiar with the Jython 2 environment, which I am told is 
based on Python 2 and NOT Python 3.


Yes, Jython 2 is currently more or less even with Python 2.7.

You are presumably writing or hosting this in a java environment, and 
you may not realize that you can call Jython code from a java class 
(calling java classes from Jython code is dead simple).  For example, I 
have some java servlet classes that invoke Jython classes and call their 
methods.  If that sounds useful for your project, I can let you know how 
to do it.


[more snips...]

--
https://mail.python.org/mailman/listinfo/python-list

Re: Help

[Mike Dewhirst via Python-list]

On 7/11/2023 9:02 am, Jason Friedman via Python-list wrote:

On Sun, Nov 5, 2023 at 1:23 PM office officce via Python-list <
python-list@python.org> wrote:


which python version is better to be used and how to make sure it works on
my window 10 because i downloaded it and it never worked so I uninstall to
do that again please can you give me the steps on how it will work perfectly


1. Download from https://python.org (not Microsoft) and always choose 
the 64-bit stable version


2. Choose the installation location as C:\Python311 (avoid the default)

4. Accept other recommended installation options especially to include 
Python on the path (if offered)


Guaranteed to work. Also, you will never have to uninstall. Install the 
next version in C:\Python312 etc


In due course, investigate virtual environments so you can work on 
projects simultaneously using different versions of Python or different 
versions of various Python libraries.


Good luck

Mike





If you are just starting out, the most recent version is 3.12 and is
probably your best choice.

When you say it never worked, can you describe in more detail what you did
and what error messages you encountered?

This mailing list does not accept screenshots.



--
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Your
email software can handle signing.



OpenPGP_signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help

[Jason Friedman via Python-list]

On Sun, Nov 5, 2023 at 1:23 PM office officce via Python-list <
python-list@python.org> wrote:

> which python version is better to be used and how to make sure it works on
> my window 10 because i downloaded it and it never worked so I uninstall to
> do that again please can you give me the steps on how it will work perfectly
>
>
If you are just starting out, the most recent version is 3.12 and is
probably your best choice.

When you say it never worked, can you describe in more detail what you did
and what error messages you encountered?

This mailing list does not accept screenshots.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help on ImportError('Error: Reinit is forbidden')

[Barry]

 On 18 May 2023, at 13:56, Jason Qian  wrote:

 
 Hi Barry,
 void handleError(const char* msg)
 {
 ...
 PyErr_Fetch(&pyExcType, &pyExcValue, &pyExcTraceback);
 PyErr_NormalizeException(&pyExcType, &pyExcValue, &pyExcTraceback);

 PyObject* str_value = PyObject_Repr(pyExcValue);
 PyObject* pyExcValueStr = PyUnicode_AsEncodedString(str_value, "utf-8",
 "Error ~");
 const char *strErrValue = PyBytes_AS_STRING(pyExcValueStr);
 //where   strErrValue   = "ImportError('Error: Reinit is forbidden')"
 ...
 }
 What we imported is a Python file which import some pyd libraries.

   Please do not top post replies.
   Ok so assume the error is correct and hunt for the code that does the
   reimport.
   You may need to set break points in you C code to find tnw caller.
   Barry

 Thanks 
 Jason 
 On Thu, May 18, 2023 at 3:53 AM Barry <[1]ba...@barrys-emacs.org> wrote:

   > On 17 May 2023, at 20:35, Jason Qian via Python-list
   <[2]python-list@python.org> wrote:
   >
   >  Hi,
   >
   >   I Need some of your help.
   >
   > I have the following C code to import *Import python.*   It works
   99% of
   > the time, but sometimes  receives  "*ImportError('Error: Reinit is
   > forbidden')*". error.
   > **We run multiple instances of the app parallelly.
   >
   > *** Python version(3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51)
   [MSC
   > v.1914 64 bit (AMD64)]
   >
   > PyObject * importPythonModule(const char* pmodName)
   > {
   >    const char* errors = NULL;
   >     int nlen = strlen(pmodName);
   >     PyObject *pName = PyUnicode_DecodeUTF8(pmodName, nlen, errors);
   >     PyObject *pModule = *PyImport_Import*(pName);
   >     Py_DECREF(pName);
   >     if (pModule == NULL) {
   >     if (*PyErr_Occurred*()) {
   >            handleError("PyImport_Import()");
   >      }
   >   }
   > }
   > void handleError(const char* msg)
   > {
   >  ...
   >  "PyImport_Import() - ImportError('Error: Reinit is forbidden')"
   > }

   You do not seem to printing out msg, you have assumed it means reinit
   it seems.
   What does msg contain when it fails?

   Barry
   >
   >
   > Thanks
   > Jason
   > --
   > [3]https://mail.python.org/mailman/listinfo/python-list
   >

References

   Visible links
   1. mailto:ba...@barrys-emacs.org
   2. mailto:python-list@python.org
   3. https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help on ImportError('Error: Reinit is forbidden')

[Jason Qian via Python-list]

Hi Barry,

void handleError(const char* msg)
{
...
PyErr_Fetch(&pyExcType, &pyExcValue, &pyExcTraceback);
PyErr_NormalizeException(&pyExcType, &pyExcValue, &pyExcTraceback);

PyObject* str_value = PyObject_Repr(pyExcValue);
PyObject* pyExcValueStr = PyUnicode_AsEncodedString(str_value, "utf-8",
"Error ~");
const char **strErrValue* = PyBytes_AS_STRING(pyExcValueStr);

//where   *strErrValue*   = "ImportError('Error: Reinit is forbidden')"
...
}

What we imported is a Python file which import some pyd libraries.


Thanks
Jason


On Thu, May 18, 2023 at 3:53 AM Barry  wrote:

>
>
> > On 17 May 2023, at 20:35, Jason Qian via Python-list <
> python-list@python.org> wrote:
> >
> >  Hi,
> >
> >   I Need some of your help.
> >
> > I have the following C code to import *Import python.*   It works 99% of
> > the time, but sometimes  receives  "*ImportError('Error: Reinit is
> > forbidden')*". error.
> > **We run multiple instances of the app parallelly.
> >
> > *** Python version(3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC
> > v.1914 64 bit (AMD64)]
> >
> > PyObject * importPythonModule(const char* pmodName)
> > {
> >const char* errors = NULL;
> > int nlen = strlen(pmodName);
> > PyObject *pName = PyUnicode_DecodeUTF8(pmodName, nlen, errors);
> > PyObject *pModule = *PyImport_Import*(pName);
> > Py_DECREF(pName);
> > if (pModule == NULL) {
> > if (*PyErr_Occurred*()) {
> >handleError("PyImport_Import()");
> >  }
> >   }
> > }
> > void handleError(const char* msg)
> > {
> >  ...
> >  "PyImport_Import() - ImportError('Error: Reinit is forbidden')"
> > }
>
> You do not seem to printing out msg, you have assumed it means reinit it
> seems.
> What does msg contain when it fails?
>
> Barry
> >
> >
> > Thanks
> > Jason
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help on ImportError('Error: Reinit is forbidden')

[Barry]

> On 17 May 2023, at 20:35, Jason Qian via Python-list  
> wrote:
> 
>  Hi,
> 
>   I Need some of your help.
> 
> I have the following C code to import *Import python.*   It works 99% of
> the time, but sometimes  receives  "*ImportError('Error: Reinit is
> forbidden')*". error.
> **We run multiple instances of the app parallelly.
> 
> *** Python version(3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC
> v.1914 64 bit (AMD64)]
> 
> PyObject * importPythonModule(const char* pmodName)
> {
>const char* errors = NULL;
> int nlen = strlen(pmodName);
> PyObject *pName = PyUnicode_DecodeUTF8(pmodName, nlen, errors);
> PyObject *pModule = *PyImport_Import*(pName);
> Py_DECREF(pName);
> if (pModule == NULL) {
> if (*PyErr_Occurred*()) {
>handleError("PyImport_Import()");
>  }
>   }
> }
> void handleError(const char* msg)
> {
>  ...
>  "PyImport_Import() - ImportError('Error: Reinit is forbidden')"
> }

You do not seem to printing out msg, you have assumed it means reinit it seems.
What does msg contain when it fails?

Barry
> 
> 
> Thanks
> Jason
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help on ctypes.POINTER for Python array

[Jason Qian via Python-list]

Awesome, thanks!

On Thu, May 11, 2023 at 1:47 PM Eryk Sun  wrote:

> On 5/11/23, Jason Qian via Python-list  wrote:
> >
> > in the Python, I have a array of string
> > var_array=["Opt1=DG","Opt1=DG2"]
> > I need to call c library and  pass var_array as parameter
> > In the   argtypes,  how do I set up ctypes.POINTER(???)  for var_array?
> >
> > func.argtypes=[ctypes.c_void_p,ctypes.c_int, ctypes.POINTER()]
> >
> > In the c code:
> > int  func (void* obj, int index,  char** opt)
>
> The argument type is ctypes.POINTER(ctypes.c_char_p), but that's not
> sufficient. It doesn't implement converting a list of str objects into
> an array of c_char_p pointers that reference byte strings. You could
> write a wrapper function that implements the conversion before calling
> func(), or you could set the argument type to a custom subclass of
> ctypes.POINTER(ctypes.c_char_p) that implements the conversion via the
> from_param() class method.
>
> https://docs.python.org/3/library/ctypes.html#ctypes._CData.from_param
>
> Here's an example of the latter.
>
> C library:
>
> #include 
>
> int
> func(void *obj, int index, char **opt)
> {
> int length;
> for (length=0; opt[length]; length++);
> if (index < 0 || index >= length) {
> return -1;
> }
> return printf("%s\n", opt[index]);
> }
>
>
> Python:
>
> import os
> import ctypes
>
> lib = ctypes.CDLL('./lib.so')
> BaseOptions = ctypes.POINTER(ctypes.c_char_p)
>
> class Options(BaseOptions):
> @classmethod
> def from_param(cls, param):
> if isinstance(param, list):
> new_param = (ctypes.c_char_p * (len(param) + 1))()
> for i, p in enumerate(param):
> new_param[i] = os.fsencode(p)
> param = new_param
> return BaseOptions.from_param(param)
>
> lib.func.argtypes = (ctypes.c_void_p, ctypes.c_int, Options)
>
>
> demo:
>
> >>> opts = ['Opt1=DG', 'Opt1=DG2']
> >>> lib.func(None, 0, opts)
> Opt1=DG
> 8
> >>> lib.func(None, 1, opts)
> Opt1=DG2
> 9
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help on ctypes.POINTER for Python array

[Eryk Sun]

On 5/11/23, Jason Qian via Python-list  wrote:
>
> in the Python, I have a array of string
> var_array=["Opt1=DG","Opt1=DG2"]
> I need to call c library and  pass var_array as parameter
> In the   argtypes,  how do I set up ctypes.POINTER(???)  for var_array?
>
> func.argtypes=[ctypes.c_void_p,ctypes.c_int, ctypes.POINTER()]
>
> In the c code:
> int  func (void* obj, int index,  char** opt)

The argument type is ctypes.POINTER(ctypes.c_char_p), but that's not
sufficient. It doesn't implement converting a list of str objects into
an array of c_char_p pointers that reference byte strings. You could
write a wrapper function that implements the conversion before calling
func(), or you could set the argument type to a custom subclass of
ctypes.POINTER(ctypes.c_char_p) that implements the conversion via the
from_param() class method.

https://docs.python.org/3/library/ctypes.html#ctypes._CData.from_param

Here's an example of the latter.

C library:

#include 

int
func(void *obj, int index, char **opt)
{
int length;
for (length=0; opt[length]; length++);
if (index < 0 || index >= length) {
return -1;
}
return printf("%s\n", opt[index]);
}


Python:

import os
import ctypes

lib = ctypes.CDLL('./lib.so')
BaseOptions = ctypes.POINTER(ctypes.c_char_p)

class Options(BaseOptions):
@classmethod
def from_param(cls, param):
if isinstance(param, list):
new_param = (ctypes.c_char_p * (len(param) + 1))()
for i, p in enumerate(param):
new_param[i] = os.fsencode(p)
param = new_param
return BaseOptions.from_param(param)

lib.func.argtypes = (ctypes.c_void_p, ctypes.c_int, Options)


demo:

>>> opts = ['Opt1=DG', 'Opt1=DG2']
>>> lib.func(None, 0, opts)
Opt1=DG
8
>>> lib.func(None, 1, opts)
Opt1=DG2
9
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help on ctypes.POINTER for Python array

[Jim Schwartz]

I’m not sure this is the shortest method, but you could set up two python 
scripts to do the same thing and convert them to c using cython. I wouldn’t be 
able to read the c scripts, but maybe you could. 

Maybe someone else has a more direct answer. 

Sent from my iPhone

> On May 11, 2023, at 10:00 AM, Jason Qian via Python-list 
>  wrote:
> 
> Hi,
> 
> Need some help,
> 
> in the Python, I have a array of string
> 
> var_array=["Opt1=DG","Opt1=DG2"]
> 
> I need to call c library and  pass var_array as parameter
> 
> In the   argtypes,  how do I set up ctypes.POINTER(???)  for var_array?
> 
> func.argtypes=[ctypes.c_void_p,ctypes.c_int, ctypes.POINTER()]
> 
> In the c code:
> 
> int  func (void* obj, int index,  char** opt)
> 
> Thanks
> Jason
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Help Request] Embedding Python in a CPP Application Responsibly & Functionally

[Dieter Maurer]

John McCardle wrote at 2023-1-25 22:31 -0500:
> ...
>1) To get the compiled Python to run independently, I have to hack
>LD_LIBRARY_PATH to get it to execute. `LD_LIBRARY_PATH=./Python-3.11.1
>./Python-3.11.1/python` .

The need to set `LD_LIBRARY_PATH` usually can be avoided via
a link time option: it tells the linker to add library path
information into the created shared object.

Read the docs to find out which option this is (I think it
was `-r` but I am not sure).

>Even when trying to execute from the same
>directory as the binary & executable, I get an error, `/python: error
>while loading shared libraries: libpython3.11.so.1.0: cannot open shared
>object file: No such file or directory`.

It might be necessary, to provide the option mentioned above
for all shared libraries involved in your final application.

Alternatively, you could try to put the shared objects
into a stadard place (searched by default).

>2) When running the C++ program that embeds Python, I see these messages
>after initializing:
>`Could not find platform independent libraries 
>Could not find platform dependent libraries `

Again: either put your installation in a standard place
or tell the Python generation process about your non-standard place.


>This is seemingly connected to some issues regarding libraries: When I
>run the Python interpreter directly, I can get some of the way through
>the process of creating a virtual environment, but it doesn't seem to
>leave me with a working pip:
>
>`$ LD_LIBRARY_PATH=./Python-3.11.1 ./Python-3.11.1/python
> >>> import venv
> >>> venv.create("./venv", with_pip=True)
>subprocess.CalledProcessError: Command
>'['/home/john/Development/7DRL/cpp_embedded_python/venv/bin/python',
>'-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit
>status 127.`

Run the command manually and see what errors this gives.

> ...

>3) I'm not sure I even need to be statically linking the interpreter.

There should be no need (if all you want in the embedding).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: help

[Chris Grace]

Hello,

The mailing list strips all images. We cannot see the error you posted. I'd
recommend posting it in text form.

What is the script you are running?

What do you expect your script to do?

On Fri, Dec 30, 2022, 3:14 PM Mor yosef  wrote:

> Hello guys,
> i install python 3.11.1, and i have google chrome so i download the
> Webdriver Chrome
> and i just add it on my c:// driver and when i add some script on my
> desktop i try to run it from the terminal and all the time i receive this
> error message : maybe you guys can help me with this one?
> when i add some random code the terminal like "python -m webbrowser
> https://www.google.com"; it's just works (it's open the website in new tab)
> Regards
> Mor yosef
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help Merging Of Separate Python Codes

[dn]

On 20/11/2022 02.20, maria python wrote:
Hello, I have these two python codes that work well separately and I 
want to combine them, but I am not sure how to do it. Also after that, I 
want to print the result in txt cvs or another file format. Do you have 
a code for that? Thank you.



Have you tried joining them together, one after the other?

Have you looked at the Python docs?
eg https://docs.python.org/3/library/csv.html

Do you need the help of a professional to write code for you?

If you are learning Python, perhaps the Tutor list will be a more 
appropriate for you...


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help, PyCharm fails to recognize my tab setting...See attached picture of the code.

[dn]

On 11/10/2022 10.48, Kevin M. Wilson via Python-list wrote:

C:\Users\kevin\PycharmProjects\Myfuturevalue\venv\Scripts\python.exe 
C:\Users\kevin\PycharmProjects\Myfuturevalue\FutureValueCal.py   File 
"C:\Users\kevin\PycharmProjects\Myfuturevalue\FutureValueCal.py", line 31    elif 
(years > 50.0) or (years < 1.0) :    ^IndentationError: expected an indented block after 
'if' statement on line 29
Process finished with exit code 1


Indentation depends upon what went before, as well as what is being 
indented 'right now'.


As you can see from the reproduction of the OP (above), any comment on 
formatting would be a guess.


Please copy-paste the entire block of the if-statement and its nested 
suites...

(this list will not pass-along images)
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python/Eyed3 MusicCDIdFrame method

[Dennis Lee Bieber]

On Mon, 6 Jun 2022 12:37:47 +0200, Dave  declaimed
the following:

>Hi,
>
>I’m trying to get the ID3 tags of an mp3 file. I trying to use the 
>MusicCDIdFrame
> method but I can’t seem to get it right. Here is a code snippet:
>
>
> import eyed3
>import eyed3.id3
>import eyed3.id3.frames
>import eyed3.id3.apple

As I understand the documentation, The last is Apple
specific/non-standard information...
https://eyed3.readthedocs.io/en/latest/eyed3.id3.html

"""
eyed3.id3.apple module

Here lies Apple frames, all of which are non-standard. All of these would
have been standard user text frames by anyone not being a bastard, on
purpose.
"""

{I'm not thrilled by the documentation -- it is basically a collection on
one-line doc-strings with absolutely no hints as to proper usage}

>  File "/Documents/Python/Test1/main.py", line 94, in 
>myCDID = myID3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')
>AttributeError: 'Mp3AudioFile' object has no attribute 'id3'

"""
 eyed3.core.load(path, tag_version=None)[source]

Loads the file identified by path and returns a concrete type of
eyed3.core.AudioFile. If path is not a file an IOError is raised. None is
returned when the file type (i.e. mime-type) is not recognized. The
following AudioFile types are supported:

eyed3.mp3.Mp3AudioFile - For mp3 audio files.

eyed3.id3.TagFile - For raw ID3 data files.
"""

eyed3.id3. would appear to be specific to non-MP3 data files.

So... I'd try the interactive environment and check each layer...

dir(myID3)

(based upon the error, there will not be a "id3" key; so try
dir(myID3.) for each key you do find).

>
>
>Any help or suggestion greatly appreciated.
>

Given this bit of source code from the documentation...

def initTag(self, version=id3.ID3_DEFAULT_VERSION):
"""Add a id3.Tag to the file (removing any existing tag if one
exists).
"""
self.tag = id3.Tag()
self.tag.version = version
self.tag.file_info = id3.FileInfo(self.path)
return self.tag

... you probably need to be looking at 
myID3.tag.

... try
dir(myID3.tag)
and see what all may appear...

IOW: the ID3 information has already been parsed into separate "tag"
fields.


-- 
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python/Eyed3 MusicCDIdFrame method

[Dave]

Thanks! That fixed it!

> On 6 Jun 2022, at 18:46, MRAB  wrote:
> 
> On 2022-06-06 11:37, Dave wrote:
>> Hi,
>> I’m trying to get the ID3 tags of an mp3 file. I trying to use the 
>> MusicCDIdFrame
>>  method but I can’t seem to get it right. Here is a code snippet:
>>  import eyed3
>> import eyed3.id3
>> import eyed3.id3.frames
>> import eyed3.id3.apple
>> import eyed3.mp3
>> myID3 = eyed3.load("/Users/Test/Life in the fast lane.mp3")
>> myTitle = myID3.tag.title
>> myArtist = myID3.tag.artist
>> myAlbum = myID3.tag.album
>> myAlbumArtist = myID3.tag.album_artist
>> myComposer = myID3.tag.composer
>> myPublisher = myID3.tag.publisher
>> myGenre = myID3.tag.genre.name
>> myCDID = myID3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')
>> When I run this, I get the following error:
>>   File "/Documents/Python/Test1/main.py", line 94, in 
>> myCDID = myID3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')
>> AttributeError: 'Mp3AudioFile' object has no attribute 'id3'
>> Any help or suggestion greatly appreciated.
> That line should be:
> 
> myCDID = eyed3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')
> 
> Also remember that some attributes might be None, e.g. 'myID3.tag.genre' 
> might be None.
> 
> Another point: it's probably not worth importing the submodules of 'eyed3'; 
> you're not gaining anything from it.
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python/Eyed3 MusicCDIdFrame method

[MRAB]

On 2022-06-06 11:37, Dave wrote:

Hi,

I’m trying to get the ID3 tags of an mp3 file. I trying to use the 
MusicCDIdFrame
  method but I can’t seem to get it right. Here is a code snippet:


  import eyed3
import eyed3.id3
import eyed3.id3.frames
import eyed3.id3.apple
import eyed3.mp3
myID3 = eyed3.load("/Users/Test/Life in the fast lane.mp3")
myTitle = myID3.tag.title
myArtist = myID3.tag.artist
myAlbum = myID3.tag.album
myAlbumArtist = myID3.tag.album_artist
myComposer = myID3.tag.composer
myPublisher = myID3.tag.publisher
myGenre = myID3.tag.genre.name
myCDID = myID3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')

When I run this, I get the following error:

   File "/Documents/Python/Test1/main.py", line 94, in 
 myCDID = myID3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')
AttributeError: 'Mp3AudioFile' object has no attribute 'id3'


Any help or suggestion greatly appreciated.


That line should be:

myCDID = eyed3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')

Also remember that some attributes might be None, e.g. 'myID3.tag.genre' 
might be None.


Another point: it's probably not worth importing the submodules of 
'eyed3'; you're not gaining anything from it.

--
https://mail.python.org/mailman/listinfo/python-list

Re: help, please, with 3.10.4 install

[Dennis Lee Bieber]

On Sat, 28 May 2022 21:11:00 -0500, Jack Gilbert <00jhen...@gmail.com>
declaimed the following:

>also, the same line: Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022,
>23:13:41) [MSC v.1929 64 bit (AMD64)] on win32 in CMD prompt
>
>for the life of me I can't figure out how to launch python??
>

Well, what did you type in that command shell to get the line you
report above? (Cut and Paste the TEXT from that command shell -- don't just
transcribe by hand). That version string is only displayed when one starts
Python in interactive mode.

-=-=-
Microsoft Windows [Version 10.0.19044.1706]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Wulfraed>python
Python ActivePython 3.8.2 (ActiveState Software Inc.) based on
 on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

-=-=-



-- 
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: help, please, with 3.10.4 install

[Mats Wichmann]

On 5/28/22 20:11, Jack Gilbert wrote:
> I downloaded 3.10.4 on a 64 bit , 8.1

> also, the same line: Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022,
> 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32 in CMD prompt
> 
> for the life of me I can't figure out how to launch python??

Sounds like you're launching it already?

In a cmd shell, type:

py


And you should be good to go.  See the page @dn pointed to.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: help, please, with 3.10.4 install

[dn]

On 29/05/2022 14.11, Jack Gilbert wrote:
> I downloaded 3.10.4 on a 64 bit , 8.1
> I can see IDLE shell 3.10.1, I see Python 3.10.4 (tags/v3.10.4:9d38120, Mar
> 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
> 
> also, the same line: Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022,
> 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32 in CMD prompt
> 
> for the life of me I can't figure out how to launch python??
> 
> I did click add to path in the install
> 
> thanks


Please advise if https://docs.python.org/3/using/windows.html does not
answer the question...
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help: Unable to find IDLE folder inside the Python Lib folder

[MRAB]

On 2022-03-05 20:36, Deji Olofinboba via Python-list wrote:


Dear Python officer,
Please I am new to programming. I have justinstalled the python 3.10.2. After 
the installation, I was able to locate thePython Shell but unable to locate 
IDLE despite checking it before downloading in the python installation folder. 
I also reinstalled Python and checked IDLE; still the IDLE is still missing. 
Please. explain to me how I can retrieve the python IDLE.
Thank You,


If you're on Windows 10, type "idle" into the search box on the taskbar.
Alternatively, look in python_folder\Lib\idlelib for idle.bat if you're 
on an older version of Windows.

--
https://mail.python.org/mailman/listinfo/python-list

Re: help

[Igor Korot]

Hi,

On Thu, Dec 9, 2021 at 10:31 AM smita  wrote:
>
>
>
>I am not able to open python on my laptop plzz help

What do you mean by saying "open python"?

Thank you.

>
>
>
>Sent from [1]Mail for Windows
>
>
>
> References
>
>Visible links
>1. https://go.microsoft.com/fwlink/?LinkId=550986
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: help

[Mats Wichmann]

On 12/9/21 03:39, smita wrote:
 


I am not able to open python on my laptop plzz help


You're going to have to provide some details.


--
https://mail.python.org/mailman/listinfo/python-list

Re: HELP - uninstall pyton error

[Michel]

give more details

OS, Python Version, Method of installation.. etc..


On 10/7/21 17:18, Almadar Plus wrote:

Could you please help me uninstall python?
see attached screenshots files
Regards

--
https://mail.python.org/mailman/listinfo/python-list

Re: HELP - uninstall pyton error

[Mats Wichmann]

On 10/7/21 10:18, Almadar Plus wrote:

Could you please help me uninstall python?
see attached screenshots files
Regards



the list doesn't pass on screenshots, so we see nothing.

you'll need to describe your problem.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help me split a string into elements

[Neil]

DFS  wrote:
> Typical cases:
>  lines = [('one\ntwo\nthree\n')]
>  print(str(lines[0]).splitlines())
>  ['one', 'two', 'three']
> 
>  lines = [('one two three\n')]
>  print(str(lines[0]).split())
>  ['one', 'two', 'three']
> 
> 
> That's the result I'm wanting, but I get data in a slightly different 
> format:
> 
> lines = [('one\ntwo\nthree\n',)]
> 
> Note the comma after the string data, but inside the paren. 
> splitlines() doesn't work on it:
> 
> print(str(lines[0]).splitlines())
> ["('one\\ntwo\\nthree\\n',)"]
> 
> 
> I've banged my head enough - can someone spot an easy fix?
> 
> Thanks

lines[0][0].splitlines()

(You have a list containing a tuple containing the string you want to
split up.)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help me split a string into elements

[DFS]

On 9/4/2021 5:55 PM, DFS wrote:

Typical cases:
  lines = [('one\ntwo\nthree\n')]
  print(str(lines[0]).splitlines())
  ['one', 'two', 'three']

  lines = [('one two three\n')]
  print(str(lines[0]).split())
  ['one', 'two', 'three']


That's the result I'm wanting, but I get data in a slightly different 
format:


lines = [('one\ntwo\nthree\n',)]

Note the comma after the string data, but inside the paren. splitlines() 
doesn't work on it:


print(str(lines[0]).splitlines())
["('one\\ntwo\\nthree\\n',)"]


I've banged my head enough - can someone spot an easy fix?

Thanks



I got it:

lines = [('one\ntwo\nthree\n',)]
print(str(lines[0][0]).splitlines())
['one', 'two', 'three']

--
https://mail.python.org/mailman/listinfo/python-list

Re: help me please. "install reppy"

[Jack DeVries]

See here for a discussion around this issue:
https://github.com/seomoz/reppy/issues/90

This project requires a C++ build environment to be setup on your computer.
The fact that your compiler is reporting that `std=c++11` as an unknown
option shows that you don't have a C++ build environment set up. As you
will see, at the end of the issue above, they link to this issue:

https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues/46

In that second issue, a user reports:
> hey all. I solved the issue by installing Build tools for visual c++ 2015
(v14). The link does not work.
> You have to look for build tools for visual studio 2017 (v15) and look at
the options in the installer.

As a side note, you might consider using this robots.txt parser from the
standard library instead. It's already installed and ready to go, and
doesn't use very-overkill C++ dependencies!

Good luck!


On Thu, Jul 22, 2021 at 8:44 PM たこしたこし  wrote:

> I get an error so please help.
> I don't know what to do.
>
> Windows 10
> Python 3.9.6
>
> from takashi in Japan
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: help me please. "install reppy"

[Jack DeVries]

Oops, forgot the link to the standard library robots.txt parser. Here are
the docs!


https://docs.python.org/3/library/urllib.robotparser.html

*sorry for the noise*

On Thu, Jul 22, 2021 at 11:34 PM Jack DeVries 
wrote:

> See here for a discussion around this issue:
> https://github.com/seomoz/reppy/issues/90
>
> This project requires a C++ build environment to be setup on your
> computer. The fact that your compiler is reporting that `std=c++11` as an
> unknown option shows that you don't have a C++ build environment set up. As
> you will see, at the end of the issue above, they link to this issue:
>
> https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues/46
>
> In that second issue, a user reports:
> > hey all. I solved the issue by installing Build tools for visual c++
> 2015 (v14). The link does not work.
> > You have to look for build tools for visual studio 2017 (v15) and look
> at the options in the installer.
>
> As a side note, you might consider using this robots.txt parser from the
> standard library instead. It's already installed and ready to go, and
> doesn't use very-overkill C++ dependencies!
>
> Good luck!
>
>
> On Thu, Jul 22, 2021 at 8:44 PM たこしたこし  wrote:
>
>> I get an error so please help.
>> I don't know what to do.
>>
>> Windows 10
>> Python 3.9.6
>>
>> from takashi in Japan
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with gaussian filter from scipy

[Ben Bacarisse]

Arak Rachael  writes:

> On Wednesday, 7 July 2021 at 12:47:40 UTC+2, Arak Rachael wrote:
>> On Wednesday, 7 July 2021 at 12:41:44 UTC+2, Ben Bacarisse wrote: 
>> > Arak Rachael  writes: 
>> > 
>> > > this time I am stuck on gaussian_filter scipy returns none instead of 
>> > > some valid gaussian filtered image, can someone help me please. 
>> > > 
>> > > Here is the full code: 
>> > > https://www.dropbox.com/s/18ylpiwmhlu5n62/test_filter_xdog.ipynb?dl=0 
>> > It might help to link to the code itself so people can look at it 
>> > without having to load up a project. 
>> > 
>> > -- 
>> > Ben.
>> Sorry I don't have this option. 
>> 
>> I loaded it in PyCharm and this is what I got: 
>> 
>> /home/yordan/anaconda3/envs/testing/bin/python 
>> /home/yordan/pycharm/plugins/python-ce/helpers/pydev/pydevd.py --multiproc 
>> --qt-support=auto --client 127.0.0.1 --port 36321 --file 
>> /home/yordan/devel/python.assignments/topgis-viz/topgisviz/xdog.py 
>> Connected to pydev debugger (build 211.7442.45) 
>> Usage: python xdog.py FILE OUTPUT 
>> 
>> I tried creating "'./textures/hatch.jpg" image, but it did not work.
>
> Sorry for taking your time, this turns to be a dum question, I needed
> to supply in file and out file in the command line.

No worries. Sometimes just posting the question make you look at it
differently. If you can post code, it's much better to do that.  Had it
been a little bit more obscure a problem, someone might have spotted it
but they may not have wanted to load a project file.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with gaussian filter from scipy

[Arak Rachael]

On Wednesday, 7 July 2021 at 12:47:40 UTC+2, Arak Rachael wrote:
> On Wednesday, 7 July 2021 at 12:41:44 UTC+2, Ben Bacarisse wrote: 
> > Arak Rachael  writes: 
> > 
> > > this time I am stuck on gaussian_filter scipy returns none instead of 
> > > some valid gaussian filtered image, can someone help me please. 
> > > 
> > > Here is the full code: 
> > > https://www.dropbox.com/s/18ylpiwmhlu5n62/test_filter_xdog.ipynb?dl=0 
> > It might help to link to the code itself so people can look at it 
> > without having to load up a project. 
> > 
> > -- 
> > Ben.
> Sorry I don't have this option. 
> 
> I loaded it in PyCharm and this is what I got: 
> 
> /home/yordan/anaconda3/envs/testing/bin/python 
> /home/yordan/pycharm/plugins/python-ce/helpers/pydev/pydevd.py --multiproc 
> --qt-support=auto --client 127.0.0.1 --port 36321 --file 
> /home/yordan/devel/python.assignments/topgis-viz/topgisviz/xdog.py 
> Connected to pydev debugger (build 211.7442.45) 
> Usage: python xdog.py FILE OUTPUT 
> 
> I tried creating "'./textures/hatch.jpg" image, but it did not work.

Sorry for taking your time, this turns to be a dum question, I needed to supply 
in file and out file in the command line.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with gaussian filter from scipy

[Ben Bacarisse]

Arak Rachael  writes:

> this time I am stuck on gaussian_filter scipy returns none instead of some 
> valid gaussian filtered image, can someone help me please.
>
> Here is the full code:
> https://www.dropbox.com/s/18ylpiwmhlu5n62/test_filter_xdog.ipynb?dl=0

It might help to link to the code itself so people can look at it
without having to load up a project.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with gaussian filter from scipy

[Arak Rachael]

On Wednesday, 7 July 2021 at 12:41:44 UTC+2, Ben Bacarisse wrote:
> Arak Rachael  writes: 
> 
> > this time I am stuck on gaussian_filter scipy returns none instead of some 
> > valid gaussian filtered image, can someone help me please. 
> > 
> > Here is the full code: 
> > https://www.dropbox.com/s/18ylpiwmhlu5n62/test_filter_xdog.ipynb?dl=0
> It might help to link to the code itself so people can look at it 
> without having to load up a project. 
> 
> -- 
> Ben.
Sorry I don't have this option.

I loaded it in PyCharm and this is what I got:

/home/yordan/anaconda3/envs/testing/bin/python 
/home/yordan/pycharm/plugins/python-ce/helpers/pydev/pydevd.py --multiproc 
--qt-support=auto --client 127.0.0.1 --port 36321 --file 
/home/yordan/devel/python.assignments/topgis-viz/topgisviz/xdog.py
Connected to pydev debugger (build 211.7442.45)
Usage: python xdog.py FILE OUTPUT

I tried creating "'./textures/hatch.jpg" image, but it did not work.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python circular error

[Arak Rachael]

On Tuesday, 15 June 2021 at 19:30:28 UTC+2, Chris Angelico wrote:
> On Wed, Jun 16, 2021 at 3:17 AM MRAB  wrote: 
> > 
> > On 2021-06-15 17:49, Chris Angelico wrote: 
> > > On Wed, Jun 16, 2021 at 2:45 AM Arak Rachael  
> > > wrote: 
> > >>
> > >> Hi to everyone, 
> > >> 
> > >> I am having a problem with this error, I created a package and uploaded 
> > >> it to Test PyPi, but I can not get it to work, can someone help me 
> > >> please? 
> > >> 
> > >> https://test.pypi.org/manage/project/videotesting/releases/' 
> > >> 
> > >> The error: 
> > >> 
> > >> /home/user/anaconda3/envs/testing/bin/python 
> > >> /home/user/devel/python.assignments/topgis-viz/topgis-test.py 
> > >> Traceback (most recent call last): 
> > >> File "/home/user/devel/python.assignments/topgis-viz/topgis-test.py", 
> > >> line 10, in  
> > >> from videotesting import downsample_and_save_npz 
> > >> File 
> > >> "/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py",
> > >>  line 7, in  
> > >> from videotesting import extract_video 
> > >> ImportError: cannot import name 'extract_video' from partially 
> > >> initialized module 'videotesting' (most likely due to a circular import) 
> > >> (/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py)
> > >>  
> > >> 
> > >
> > > Hard to diagnose without the source code, but I'm thinking that your 
> > > __init__.py is probably trying to import from elsewhere in the 
> > > package? If so, try "from . import extract_video" instead. 
> > > 
> > Well, the traceback says that videotesting/__init__.py has: 
> > from videotesting import extract_video 
> > 
> > so it is trying to import from itself during initialisation.
> Yes, but what we can't tell is what the intention is. It could be 
> trying to import from its own package (in which case my suggestion 
> would be correct), or it could be trying to import from something 
> completely different (in which case the solution is to rename 
> something to avoid the conflict). Or it could be something else 
> entirely. 
> 
> ChrisA
Thanks all!


I have __init__.py in the create and uploaded package and I have __init_.py in 
the new package, which is not ready yet. Its 2 projects, but I am using project 
1 into project 2.

Here is the first package as a .zip
https://www.dropbox.com/s/lkz0qf4mq9afpaz/video-testing-package1.zip?dl=0

Here is the second project in which I try to use the first one:
https://www.dropbox.com/s/sxjpip23619fven/topgis-viz-package2.zip?dl=0
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python circular error

[Arak Rachael]

On Wednesday, 16 June 2021 at 10:32:50 UTC+2, Arak Rachael wrote:
> On Tuesday, 15 June 2021 at 19:30:28 UTC+2, Chris Angelico wrote: 
> > On Wed, Jun 16, 2021 at 3:17 AM MRAB  wrote: 
> > > 
> > > On 2021-06-15 17:49, Chris Angelico wrote: 
> > > > On Wed, Jun 16, 2021 at 2:45 AM Arak Rachael  
> > > > wrote: 
> > > >> 
> > > >> Hi to everyone, 
> > > >> 
> > > >> I am having a problem with this error, I created a package and 
> > > >> uploaded it to Test PyPi, but I can not get it to work, can someone 
> > > >> help me please? 
> > > >> 
> > > >> https://test.pypi.org/manage/project/videotesting/releases/' 
> > > >> 
> > > >> The error: 
> > > >> 
> > > >> /home/user/anaconda3/envs/testing/bin/python 
> > > >> /home/user/devel/python.assignments/topgis-viz/topgis-test.py 
> > > >> Traceback (most recent call last): 
> > > >> File "/home/user/devel/python.assignments/topgis-viz/topgis-test.py", 
> > > >> line 10, in  
> > > >> from videotesting import downsample_and_save_npz 
> > > >> File 
> > > >> "/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py",
> > > >>  line 7, in  
> > > >> from videotesting import extract_video 
> > > >> ImportError: cannot import name 'extract_video' from partially 
> > > >> initialized module 'videotesting' (most likely due to a circular 
> > > >> import) 
> > > >> (/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py)
> > > >>  
> > > >> 
> > > > 
> > > > Hard to diagnose without the source code, but I'm thinking that your 
> > > > __init__.py is probably trying to import from elsewhere in the 
> > > > package? If so, try "from . import extract_video" instead. 
> > > > 
> > > Well, the traceback says that videotesting/__init__.py has: 
> > > from videotesting import extract_video 
> > > 
> > > so it is trying to import from itself during initialisation. 
> > Yes, but what we can't tell is what the intention is. It could be 
> > trying to import from its own package (in which case my suggestion 
> > would be correct), or it could be trying to import from something 
> > completely different (in which case the solution is to rename 
> > something to avoid the conflict). Or it could be something else 
> > entirely. 
> > 
> > ChrisA
> Thanks all! 
> 
> 
> I have __init__.py in the create and uploaded package and I have __init_.py 
> in the new package, which is not ready yet. Its 2 projects, but I am using 
> project 1 into project 2. 
> 
> Here is the first package as a .zip 
> https://www.dropbox.com/s/lkz0qf4mq9afpaz/video-testing-package1.zip?dl=0 
> 
> Here is the second project in which I try to use the first one: 
> https://www.dropbox.com/s/sxjpip23619fven/topgis-viz-package2.zip?dl=0

The problems were 2:

1. This has to be in __init__.py:

import pipreqs
import cv2
import numpy
import os
import PIL
#import videotesting
from .videotest import extract_video
from .videotest import resize_and_grayscale
from .videotest import add_progress_bar
from .videotest import downsample_and_save_npz

the dots before videotest are critical

2. The module is named videotest as the file videotest.py, not videotesting
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python circular error

[Chris Angelico]

On Wed, Jun 16, 2021 at 3:17 AM MRAB  wrote:
>
> On 2021-06-15 17:49, Chris Angelico wrote:
> > On Wed, Jun 16, 2021 at 2:45 AM Arak Rachael  
> > wrote:
> >>
> >> Hi to everyone,
> >>
> >> I am having a problem with this error, I created a package and uploaded it 
> >> to Test PyPi, but I can not get it to work, can someone help me please?
> >>
> >> https://test.pypi.org/manage/project/videotesting/releases/'
> >>
> >> The error:
> >>
> >> /home/user/anaconda3/envs/testing/bin/python 
> >> /home/user/devel/python.assignments/topgis-viz/topgis-test.py
> >> Traceback (most recent call last):
> >>   File "/home/user/devel/python.assignments/topgis-viz/topgis-test.py", 
> >> line 10, in 
> >> from videotesting import downsample_and_save_npz
> >>   File 
> >> "/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py",
> >>  line 7, in 
> >> from videotesting import extract_video
> >> ImportError: cannot import name 'extract_video' from partially initialized 
> >> module 'videotesting' (most likely due to a circular import) 
> >> (/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py)
> >>
> >
> > Hard to diagnose without the source code, but I'm thinking that your
> > __init__.py is probably trying to import from elsewhere in the
> > package? If so, try "from . import extract_video" instead.
> >
> Well, the traceback says that videotesting/__init__.py has:
>  from videotesting import extract_video
>
> so it is trying to import from itself during initialisation.

Yes, but what we can't tell is what the intention is. It could be
trying to import from its own package (in which case my suggestion
would be correct), or it could be trying to import from something
completely different (in which case the solution is to rename
something to avoid the conflict). Or it could be something else
entirely.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python circular error

[MRAB]

On 2021-06-15 17:49, Chris Angelico wrote:

On Wed, Jun 16, 2021 at 2:45 AM Arak Rachael  wrote:


Hi to everyone,

I am having a problem with this error, I created a package and uploaded it to 
Test PyPi, but I can not get it to work, can someone help me please?

https://test.pypi.org/manage/project/videotesting/releases/'

The error:

/home/user/anaconda3/envs/testing/bin/python 
/home/user/devel/python.assignments/topgis-viz/topgis-test.py
Traceback (most recent call last):
  File "/home/user/devel/python.assignments/topgis-viz/topgis-test.py", line 10, in 

from videotesting import downsample_and_save_npz
  File 
"/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py",
 line 7, in 
from videotesting import extract_video
ImportError: cannot import name 'extract_video' from partially initialized 
module 'videotesting' (most likely due to a circular import) 
(/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py)



Hard to diagnose without the source code, but I'm thinking that your
__init__.py is probably trying to import from elsewhere in the
package? If so, try "from . import extract_video" instead.


Well, the traceback says that videotesting/__init__.py has:
from videotesting import extract_video

so it is trying to import from itself during initialisation.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help with Python circular error

[Chris Angelico]

On Wed, Jun 16, 2021 at 2:45 AM Arak Rachael  wrote:
>
> Hi to everyone,
>
> I am having a problem with this error, I created a package and uploaded it to 
> Test PyPi, but I can not get it to work, can someone help me please?
>
> https://test.pypi.org/manage/project/videotesting/releases/'
>
> The error:
>
> /home/user/anaconda3/envs/testing/bin/python 
> /home/user/devel/python.assignments/topgis-viz/topgis-test.py
> Traceback (most recent call last):
>   File "/home/user/devel/python.assignments/topgis-viz/topgis-test.py", line 
> 10, in 
> from videotesting import downsample_and_save_npz
>   File 
> "/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py",
>  line 7, in 
> from videotesting import extract_video
> ImportError: cannot import name 'extract_video' from partially initialized 
> module 'videotesting' (most likely due to a circular import) 
> (/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/videotesting/__init__.py)
>

Hard to diagnose without the source code, but I'm thinking that your
__init__.py is probably trying to import from elsewhere in the
package? If so, try "from . import extract_video" instead.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: HELP Please, Python Program Help

[dn via Python-list]

On 10/04/2021 22.57, Joseph Roffey wrote:
> Hi, Im looking for some help with my program, I have been set a task to make 
> a Strain Calculator. I need it to input two numbers, choosing either Metres 
> or Inches for the 'Change in Length' divided by the 'Original Length' which 
> can also be in Metres or Inches, the out put number also needs to give an 
> option for the answer to be in metres or inches.
> 
> this is what i have come up with so far...

What is the problem?


> txt = "Strain Calculator"
> x = txt.title()

what is the above/


> print(x)
> 
> # This function divides two numbers
> def divide(x, y):
>   return x / y

Why use a function instead of operating in-line?

-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help Please

[D'Arcy Cain]

On 2021-03-26 12:42 p.m., Igor Korot wrote:

On top of that - usual stanza applies:

1. OS - Windows, Linux, Mac?
2. OS version?
3. Python version?
4. Are you able to run python interpretor?
5. Socks version you are trying to install?
6. Was install successful?


7. Use a subject that describes the actual issue.

--
D'Arcy J.M. Cain
Vybe Networks Inc.
A unit of Excelsior Solutions Corporation - Propelling Business Forward
http://www.VybeNetworks.com/
IM:da...@vybenetworks.com VoIP: sip:da...@vybenetworks.com
--
https://mail.python.org/mailman/listinfo/python-list