Re: How to write partial of a buffer which was returned from a C function to a file?

[Chris Angelico]

On Thu, Apr 12, 2018 at 2:16 PM,   wrote:
> This C function returns a buffer which I declared it as a ctypes.c_char_p. 
> The buffer has size 0x1 bytes long and the valid data may vary from a few 
> bytes to the whole size.
>
> In every call I know how much the valid data size is, but I suppose I can't 
> use slice to get it because there may be zero byte in it. What to do?
>

You suppose? Or have you tested it?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

How to write partial of a buffer which was returned from a C function to a file?

[jfong]

This C function returns a buffer which I declared it as a ctypes.c_char_p. The 
buffer has size 0x1 bytes long and the valid data may vary from a few bytes 
to the whole size.

In every call I know how much the valid data size is, but I suppose I can't use 
slice to get it because there may be zero byte in it. What to do?


Best Regards,
Jach Fong
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

[cuddlycaveman]

I’m replying to your post on January 28th
Nice carefully chosen non random numbers  Steven D'Aprano.
Was just doing what you asked, but you don’t remember 😂😂😂
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Filtering computer.lang.python

[Skip Montanaro]

> I fetch comp.lang.python from eternal.september with leafnode, and after
> 30 years of Usenet I recently had to install a news filter to remove the
> garbage.  After the initial flurry the filter doesn't need much updating,
> but here's why it's necessary:
...
> for totals of 2168 fetched and 4384 killed; that is, the group
> is now 2/3 spam and the volume doesn't seem to be decreasing.
> I don't understand why other groups gatewayed to Google Groups
> aren't spammed, but from a limited sample they don't seem to be.

Just a thought, but you could maybe use SpamBayes to write your own
spam filter for comp.lang.python. The gate_news program on
mail.python.org does just that, using the training database I
maintain. If anyone is interested, contact me. I can help set you up
with the gate_news code.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Filtering computer.lang.python

[Mark Lawrence]

On 10/04/18 21:06, C W Rose via Python-list wrote:

Thomas Jollans  wrote:


Welcome to python-list/comp.lang.python!

This isn't originally a Google group. Google just mirrors the old USENET
group, which is awash with spam.

There is also a mailing list version of this group (posts are mirrored
both ways) at https://mail.python.org/mailman/listinfo/python-list

The mailing list has proper spam filtering and some moderation. None (or
barely any) of the regulars use Google Groups. Some people use USENET
directly and maintain their own extensive filtering regime to make it
readable. Probably most of us use the mailing list, because it's just so
much nicer!

-- Thomas


I fetch comp.lang.python from eternal.september with leafnode, and after
30 years of Usenet I recently had to install a news filter to remove the
garbage.  After the initial flurry the filter doesn't need much updating,
but here's why it's necessary:

 Feb  1 comp.lang.python: 39 articles fetched 73 killed
 Feb  2 comp.lang.python: 43 articles fetched 57 killed
 Feb  3 comp.lang.python: 19 articles fetched 108 killed
 Feb  4 comp.lang.python: 36 articles fetched 122 killed
 Feb  5 comp.lang.python: 45 articles fetched 79 killed
 Feb  6 comp.lang.python: 68 articles fetched 93 killed
 Feb  7 comp.lang.python: 32 articles fetched 118 killed
 Feb  8 comp.lang.python: 41 articles fetched 100 killed
 Feb  9 comp.lang.python: 47 articles fetched 201 killed
 Feb 10 comp.lang.python: 44 articles fetched 137 killed
 Feb 11 comp.lang.python: 36 articles fetched 130 killed
 Feb 12 comp.lang.python: 13 articles fetched 79 killed
 Feb 13 comp.lang.python: 18 articles fetched 65 killed
 Feb 14 comp.lang.python: 34 articles fetched 72 killed
 Feb 15 comp.lang.python: 15 articles fetched 63 killed
 Feb 16 comp.lang.python: 14 articles fetched 72 killed
 Feb 17 comp.lang.python: 50 articles fetched 62 killed
 Feb 18 comp.lang.python: 37 articles fetched 35 killed
 Feb 19 comp.lang.python: 58 articles fetched 57 killed
 Feb 20 comp.lang.python: 71 articles fetched 46 killed
 Feb 21 comp.lang.python: 63 articles fetched 95 killed
 Feb 22 comp.lang.python: 35 articles fetched 91 killed
 Feb 23 comp.lang.python: 38 articles fetched 91 killed
 Feb 25 comp.lang.python: 51 articles fetched 153 killed
 Feb 26 comp.lang.python: 70 articles fetched 89 killed
 Feb 27 comp.lang.python: 64 articles fetched 68 killed
 Mar  1 comp.lang.python: 98 articles fetched 133 killed
 Mar  2 comp.lang.python: 65 articles fetched 110 killed
 Mar  3 comp.lang.python: 38 articles fetched 113 killed
 Mar  4 comp.lang.python: 31 articles fetched 16 killed
 Mar  5 comp.lang.python: 51 articles fetched 104 killed
 Mar  6 comp.lang.python: 46 articles fetched 40 killed
 Mar  7 comp.lang.python: 18 articles fetched 71 killed
 Mar  8 comp.lang.python: 44 articles fetched 85 killed
 Mar  9 comp.lang.python: 43 articles fetched 65 killed
 Mar 10 comp.lang.python: 16 articles fetched 39 killed
 Mar 11 comp.lang.python: 25 articles fetched 59 killed
 Mar 13 comp.lang.python: 43 articles fetched 186 killed
 Mar 14 comp.lang.python: 20 articles fetched 76 killed
 Mar 15 comp.lang.python: 32 articles fetched 33 killed
 Mar 16 comp.lang.python: 16 articles fetched 79 killed
 Mar 17 comp.lang.python: 11 articles fetched 69 killed
 Mar 18 comp.lang.python: 4  articles fetched 70 killed
 Mar 19 comp.lang.python: 10 articles fetched 67 killed
 Mar 20 comp.lang.python: 42 articles fetched 33 killed
 Mar 21 comp.lang.python: 43 articles fetched 77 killed
 Mar 22 comp.lang.python: 14 articles fetched 35 killed
 Mar 23 comp.lang.python: 37 articles fetched 39 killed
 Mar 26 comp.lang.python: 139 articles fetched 144 killed
 Mar 27 comp.lang.python: 83 articles fetched 86 killed
 Mar 28 comp.lang.python: 39 articles fetched 45 killed
 Mar 29 comp.lang.python: 12 articles fetched 29 killed
 Mar 31 comp.lang.python: 67 articles fetched 125 killed

for totals of 2168 fetched and 4384 killed; that is, the group
is now 2/3 spam and the volume doesn't seem to be decreasing.
I don't understand why other groups gatewayed to Google Groups
aren't spammed, but from a limited sample they don't seem to be.

Will



I have recently given up killing all the crap directly on gg as I can't 
be bothered any more.  That is I would go onto gg and directly mark all 
the spam as spam.  Is this coincidence?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: beacons and geofences

[ElChino]

Rafal Sikora wrote:


Hi! I want users’ devices to be able to monitor the maximum amount of POIs at
once (geo-fences/beacons) and I need to prepare an algorithm solution for

>  monitoring the POIs. How should it be implemented in Python?


What? You'll have to describe the problem in more details
if you want any sensible answers.



--
https://mail.python.org/mailman/listinfo/python-list

Re: Pandas, create new column if previous column(s) are not in [None, '', np.nan]

[Albert-Jan Roskam]

On Apr 11, 2018 20:52, zljubi...@gmail.com wrote:
>
> I have a dataframe:
>
> import pandas as pd
> import numpy as np
>
> df = pd.DataFrame( { 'A' : ['a', 'b', '', None, np.nan],
>  'B'  : [None, np.nan, 'a', 'b', '']})
>
>   A B
> 0 a  None
> 1 b   NaN
> 2   a
> 3  None b
> 4   NaN
>
>
> I would like to create column C in the following way:
> column C = column B if column B is not in [None, '', np.nan]
> else column A
>
> How to do that?
>
> I tried:
>
> df['C'] = df[['A', 'B']].apply(lambda x: x[1] if x[1] in [None, '', np.nan] 
> else x[0])
>
> but I got all np.nan's.

This is another approach:
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.replace.html
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] multicore/cpu history

[Grant Edwards]

On 2018-03-25, Steven D'Aprano  wrote:

> Not really. With multiple CPUs, you have the option of running two 
> distinct OSes in isolation, not merely virtual machines but actual 
> distinct machines in the same box.

Not on any of the multi-CPU motherboards I ever worked with.  The CPUs
shared SDRAM and used the same physical address space.  They both saw
the same PCI/ISA buses and all other peripherals. There was no way you
could run two different OSes without some sort of hypervisor -- there
was no practical difference between them and a modern multi-core CPU.

-- 
Grant Edwards   grant.b.edwardsYow! All of life is a blur
  at   of Republicans and meat!
  gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] multicore/cpu history

[Peter J. Holzer]

On 2018-03-25 22:52:59 +, Steven D'Aprano wrote:
> On Sun, 25 Mar 2018 23:29:07 +0200, Peter J. Holzer wrote:
> >> >> By the way, multiple CPU machines are different from CPUs with
> >> >> multiple cores:
> >> >>
> >> >> http://smallbusiness.chron.com/multiple-cpu-vs-multicore-33195.html
> >> > 
> >> > Yeah, it was always "multiple CPUs", not "multiple cores" when I was
> >> > growing up.
> > 
> > Yes, but the difference is only an implementation detail.
> 
> Not really. With multiple CPUs, you have the option of running two 
> distinct OSes in isolation, not merely virtual machines but actual 
> distinct machines in the same box.

Not in general, no. There may be hardware architectures which allow this
(if I remember correctly, hardware partitioning on HP and IBM unix
machines in the early noughties worked like this), but on a typical PC
motherboard this wouldn't work: There is a lot of shared hardware
outside of the CPUs, and two OSes running on different processors would
have to be aware of each other to avoid stepping on each other's toes.
And if they can do that, they can also do it on two cores of the same
CPU.

In a normal SMP system, there is no real difference between having 2
8-core processors and 1 16-core processor from the OS's point of view.
The scheduler cares about it because caches and NUMA may make migrating
a process from one core to another more expensive depending on where
that other core is physically, but otherwise a core is processor.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pandas, create new column if previous column(s) are not in [None, '', np.nan]

[codewizard]

On Wednesday, April 11, 2018 at 2:49:01 PM UTC-4, zlju...@gmail.com wrote:
> I have a dataframe:
> 
> import pandas as pd
> import numpy as np
> 
> df = pd.DataFrame( { 'A' : ['a', 'b', '', None, np.nan],
>  'B'  : [None, np.nan, 'a', 'b', '']})
> 
>   A B
> 0 a  None
> 1 b   NaN
> 2   a
> 3  None b
> 4   NaN  
> 
> 
> I would like to create column C in the following way:
> column C = column B if column B is not in [None, '', np.nan]
> else column A
> 
> How to do that?
> 
> I tried:
> 
> df['C'] = df[['A', 'B']].apply(lambda x: x[1] if x[1] in [None, '', np.nan] 
> else x[0])
> 
> but I got all np.nan's.
> 
> Where am I wrong?
> 
> I am expecting to get column C as ['a', 'b', 'a', 'b', NaN]
> 
> Regards.

Try this:

df['C'] = df['B'].where(df['B'], other=df['A'])

Regards,
Igor.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pandas, create new column if previous column(s) are not in [None, '', np.nan]

[José María Mateos]

On Wed, Apr 11, 2018, at 14:48, zljubi...@gmail.com wrote:
> I have a dataframe:
> [...]

This seems to work:

df1 = pd.DataFrame( { 'A' : ['a', 'b', '', None, np.nan],
  'B'  : [None, np.nan, 'a', 
'b', '']})
df1['C'] = df1[['A', 'B']].apply(lambda x: x[0] if x[1] in [None, '', np.nan] 
else x[1], axis = 1)

Two notes:

- Do apply() on axis = 1, so you process every row.
- You lambda function wasn't entirely correct, if I understood what you wanted 
to do.

Cheers,

-- 
José María (Chema) Mateos
https://rinzewind.org/blog-es || https://rinzewind.org/blog-en
-- 
https://mail.python.org/mailman/listinfo/python-list

Pandas, create new column if previous column(s) are not in [None, '', np.nan]

[zljubisic]

I have a dataframe:

import pandas as pd
import numpy as np

df = pd.DataFrame( { 'A' : ['a', 'b', '', None, np.nan],
 'B'  : [None, np.nan, 'a', 'b', '']})

  A B
0 a  None
1 b   NaN
2   a
3  None b
4   NaN  


I would like to create column C in the following way:
column C = column B if column B is not in [None, '', np.nan]
else column A

How to do that?

I tried:

df['C'] = df[['A', 'B']].apply(lambda x: x[1] if x[1] in [None, '', np.nan] 
else x[0])

but I got all np.nan's.

Where am I wrong?

I am expecting to get column C as ['a', 'b', 'a', 'b', NaN]

Regards.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Levenberg-Marquardt Algorithm

[tejaswi prakash]

I am sorry, but I thought Levenberg marquardt was used quite  bit in Image
registration. Computing/refining homographies between two related views for
instance.

On Wed, Apr 11, 2018 at 12:49 PM, Christian Gollwitzer 
wrote:

> Am 11.04.18 um 08:38 schrieb Priya Singh:
>
>> I have two 2D arrays one R and another T (which is also a 2D array).
>> Do you know how can I fit T with R in order to find central
>> coordinate x0,y0 for T relative to R???
>>
>> So the main question is do you know in python how can I fit two 2D arrays
>> to find
>> x0,y0 for one array relative to other. I shall use LM fit in python. But
>> for fitting, I need to have some fittable model but here I am having only
>> two 2D arrays. I know simple cross-correlation would have solved my problem
>> but I have been instructed to do fitting using one array to other.
>>
>
>
> The request is nonsense. LM fits an analytical model to data, if you don't
> have an analytical model, you need another tool. Cross correlation is
> widely used and works well for many such tasks.
>
> In principle you could also interpolate the one array to new coordinates,
> e.g. using scipy.ndimage.interpolation.shift, and minimize the sum of
> squared differences. But still LM is the wrong tool here, it would get
> trapped in local minima soon, and it uses derivatives. Look for "image
> registration" to find typical algorithms used in this context.
>
>
>
> Christian
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Filtering computer.lang.python

[C W Rose via Python-list]

Thomas Jollans  wrote:
> 
> Welcome to python-list/comp.lang.python!
> 
> This isn't originally a Google group. Google just mirrors the old USENET
> group, which is awash with spam.
> 
> There is also a mailing list version of this group (posts are mirrored
> both ways) at https://mail.python.org/mailman/listinfo/python-list
> 
> The mailing list has proper spam filtering and some moderation. None (or
> barely any) of the regulars use Google Groups. Some people use USENET
> directly and maintain their own extensive filtering regime to make it
> readable. Probably most of us use the mailing list, because it's just so
> much nicer!
> 
> -- Thomas

I fetch comp.lang.python from eternal.september with leafnode, and after
30 years of Usenet I recently had to install a news filter to remove the
garbage.  After the initial flurry the filter doesn't need much updating,
but here's why it's necessary:

Feb  1 comp.lang.python: 39 articles fetched 73 killed
Feb  2 comp.lang.python: 43 articles fetched 57 killed
Feb  3 comp.lang.python: 19 articles fetched 108 killed
Feb  4 comp.lang.python: 36 articles fetched 122 killed
Feb  5 comp.lang.python: 45 articles fetched 79 killed
Feb  6 comp.lang.python: 68 articles fetched 93 killed
Feb  7 comp.lang.python: 32 articles fetched 118 killed
Feb  8 comp.lang.python: 41 articles fetched 100 killed
Feb  9 comp.lang.python: 47 articles fetched 201 killed
Feb 10 comp.lang.python: 44 articles fetched 137 killed
Feb 11 comp.lang.python: 36 articles fetched 130 killed
Feb 12 comp.lang.python: 13 articles fetched 79 killed
Feb 13 comp.lang.python: 18 articles fetched 65 killed
Feb 14 comp.lang.python: 34 articles fetched 72 killed
Feb 15 comp.lang.python: 15 articles fetched 63 killed
Feb 16 comp.lang.python: 14 articles fetched 72 killed
Feb 17 comp.lang.python: 50 articles fetched 62 killed
Feb 18 comp.lang.python: 37 articles fetched 35 killed
Feb 19 comp.lang.python: 58 articles fetched 57 killed
Feb 20 comp.lang.python: 71 articles fetched 46 killed
Feb 21 comp.lang.python: 63 articles fetched 95 killed
Feb 22 comp.lang.python: 35 articles fetched 91 killed
Feb 23 comp.lang.python: 38 articles fetched 91 killed
Feb 25 comp.lang.python: 51 articles fetched 153 killed
Feb 26 comp.lang.python: 70 articles fetched 89 killed
Feb 27 comp.lang.python: 64 articles fetched 68 killed
Mar  1 comp.lang.python: 98 articles fetched 133 killed
Mar  2 comp.lang.python: 65 articles fetched 110 killed
Mar  3 comp.lang.python: 38 articles fetched 113 killed
Mar  4 comp.lang.python: 31 articles fetched 16 killed
Mar  5 comp.lang.python: 51 articles fetched 104 killed
Mar  6 comp.lang.python: 46 articles fetched 40 killed
Mar  7 comp.lang.python: 18 articles fetched 71 killed
Mar  8 comp.lang.python: 44 articles fetched 85 killed
Mar  9 comp.lang.python: 43 articles fetched 65 killed
Mar 10 comp.lang.python: 16 articles fetched 39 killed
Mar 11 comp.lang.python: 25 articles fetched 59 killed
Mar 13 comp.lang.python: 43 articles fetched 186 killed
Mar 14 comp.lang.python: 20 articles fetched 76 killed
Mar 15 comp.lang.python: 32 articles fetched 33 killed
Mar 16 comp.lang.python: 16 articles fetched 79 killed
Mar 17 comp.lang.python: 11 articles fetched 69 killed
Mar 18 comp.lang.python: 4  articles fetched 70 killed
Mar 19 comp.lang.python: 10 articles fetched 67 killed
Mar 20 comp.lang.python: 42 articles fetched 33 killed
Mar 21 comp.lang.python: 43 articles fetched 77 killed
Mar 22 comp.lang.python: 14 articles fetched 35 killed
Mar 23 comp.lang.python: 37 articles fetched 39 killed
Mar 26 comp.lang.python: 139 articles fetched 144 killed
Mar 27 comp.lang.python: 83 articles fetched 86 killed
Mar 28 comp.lang.python: 39 articles fetched 45 killed
Mar 29 comp.lang.python: 12 articles fetched 29 killed
Mar 31 comp.lang.python: 67 articles fetched 125 killed

for totals of 2168 fetched and 4384 killed; that is, the group
is now 2/3 spam and the volume doesn't seem to be decreasing.
I don't understand why other groups gatewayed to Google Groups
aren't spammed, but from a limited sample they don't seem to be.

Will

-- 
"That which is hateful to you, do not do to your fellow.  That is the
 whole Torah; the rest is commentary."
  -- Hillel the Elder

-- 
https://mail.python.org/mailman/listinfo/python-list

beacons and geofences

[Rafal Sikora]

Hi! I want users’ devices to be able to monitor the maximum amount of POIs at 
once (geo-fences/beacons) and I need to prepare an algorithm solution for 
monitoring the POIs. How should it be implemented in Python?
-- 
https://mail.python.org/mailman/listinfo/python-list

Levenberg-Marquardt Algorithm

[Priya Singh]

Good morning. 
I need some suggestion from you if you have encountered this problem ever.

I have two 2D arrays one R and another T (which is also a 2D array).
Do you know how can I fit T with R in order to find central
coordinate x0,y0 for T relative to R???

So the main question is do you know in python how can I fit two 2D arrays to 
find 
x0,y0 for one array relative to other. I shall use LM fit in python. But for 
fitting, I need to have some fittable model but here I am having only two 2D 
arrays. I know simple cross-correlation would have solved my problem but I have 
been instructed to do fitting using one array to other.

Any comment or suggestion would be helpful. Thanks in advanced.

Cheers!!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

[Steven D'Aprano]

On Tue, 10 Apr 2018 23:36:27 -0700, cuddlycaveman wrote:

[snip a number of carefully chosen, non-random numbers shown in binary]

> Don’t know if that helps


Helps what?

With no context, we don't know who you are replying to, what they asked, 
or why you think this is helpful.

According to my archives, such as they are, you appear to be responding 
to a post made in either July 2016 or October 2017. 


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ANN: PyDDF Python Spring Sprint 2018

[Karsten Hilbert]

>  * mxDateTime Portierung auf Python 3.6

+1  !!

Karsten Hilbert
-- 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Levenberg-Marquardt Algorithm

[Christian Gollwitzer]

Am 11.04.18 um 08:38 schrieb Priya Singh:

I have two 2D arrays one R and another T (which is also a 2D array).
Do you know how can I fit T with R in order to find central
coordinate x0,y0 for T relative to R???

So the main question is do you know in python how can I fit two 2D arrays to 
find
x0,y0 for one array relative to other. I shall use LM fit in python. But for 
fitting, I need to have some fittable model but here I am having only two 2D 
arrays. I know simple cross-correlation would have solved my problem but I have 
been instructed to do fitting using one array to other.



The request is nonsense. LM fits an analytical model to data, if you 
don't have an analytical model, you need another tool. Cross correlation 
is widely used and works well for many such tasks.


In principle you could also interpolate the one array to new 
coordinates, e.g. using scipy.ndimage.interpolation.shift, and minimize 
the sum of squared differences. But still LM is the wrong tool here, it 
would get trapped in local minima soon, and it uses derivatives. Look 
for "image registration" to find typical algorithms used in this context.




Christian

--
https://mail.python.org/mailman/listinfo/python-list