[ewg] Re: problems with ofed-1.2.c chelsio low level driver

[Michael S. Tsirkin]

> Quoting Steve Wise <[EMAIL PROTECTED]>:
> Subject: problems with ofed-1.2.c chelsio low level driver
> 
> Michael,
> 
> I've discovered that I was really testing the ofed-1.2 cxgb3.ko module 
> in my testing 1.2.c regression test.  Upon getting a complete 1.2.c 
> installed, I'm seeing crashes with the chelsio driver.  I think there 
> are some patches that went into the ofed-1.2 branch, that did not get 
> merged into ofed-1.2.c.   The ofed-1.2 branch works, but 1.2.c is broken.
> 
> I'm investigating now, but we'll need to fix this before shipping 
> 1.2.c/1.2.5.

I went through list of patches again and I don't see anything missing.
Please send fixes, if any, to Vlad by this weekend, so Vlad can integrate
them on Sunday.

-- 
MST
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: RFCv2: SRC API

[Michael S. Tsirkin]

> Quoting Tang, Changqing <[EMAIL PROTECTED]>:
> Subject: RE: RFCv2: SRC API
> 
> 
> > 
> > > Is there any performance worry to let j2(the first job on a 
> > node) to 
> > > do all the "work" ?
> > 
> > How do you mean?
> 
> I mean that j2 has all the QP connections with all other remote jobs, so
> j2 does not save anything compared with non-SRC case, can this affect
> j2's performance ?
> 
> I have another question, when using SRQ, or SRC+SRQ, when a completion
> is returned by ibv_poll_cq(),  is there an easy way to find who sent
> this message to me ? 'struct ibv_wc' does not say much about the message
> source rank. The only field is 'src_qp', but I can not find any document
> about how to use it.

There's also qp_num.

> Is there any way to return some kind of "source context" ?
> 
> Or do you think this is application's responsibility to figure out
> source rank from the message itself ? I hope the source rank can be
> identified before parsing the message.
> 
> Thanks.

How about using send with immediate, and sticking the rank in
the immediate data?

-- 
MST
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] RE: RFCv2: SRC API

[Tang, Changqing]

> 
> > Is there any performance worry to let j2(the first job on a 
> node) to 
> > do all the "work" ?
> 
> How do you mean?

I mean that j2 has all the QP connections with all other remote jobs, so
j2 does not save anything compared with non-SRC case, can this affect
j2's performance ?

I have another question, when using SRQ, or SRC+SRQ, when a completion
is returned by ibv_poll_cq(),  is there an easy way to find who sent
this message to me ? 'struct ibv_wc' does not say much about the message
source rank. The only field is 'src_qp', but I can not find any document
about how to use it.
Is there any way to return some kind of "source context" ?

Or do you think this is application's responsibility to figure out
source rank from the message itself ? I hope the source rank can be
identified before parsing the message.

Thanks.

--CQ



> 
> > What is the latency of SRC+SRQ ?
> 
> I'd expect it to be more or less the same as regular SRQ.
> 
> --
> MST
> 
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[David Miller]

From: Andi Kleen <[EMAIL PROTECTED]>
Date: 08 Aug 2007 14:38:10 +0200

> Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > + val, reg_index, addr, addr+4); */
> > > + writel(cpu_to_le32(reg_index), addr);
> > > + writel(cpu_to_le32(val),(u8 *)addr + 4);
> > 
> > wrong -- endian conversion macros not needed with writel()
> 
> Are you sure? I don't think that's true. e.g. powerpc writel 
> doesn't convert endian

raw_writel() doesn't, but writel() does.

Why not look at the code, as I did, instead of "think"ing? :-)
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] [PATCH 11/14] nes: OpenFabrics kernel verbs

[Roland Dreier]

 > +static struct ib_mw *nes_alloc_mw(struct ib_pd *ibpd) {

 > +get_random_bytes(&next_stag_index, sizeof(next_stag_index));

Could this use up a lot of entropy?  Is random32() sufficient?

 > +stag_key = (u8)next_stag_index;

I don't think this cast is needed.

 > +if (ret) {
 > +return (ERR_PTR(ret));
 > +}

Don't need braces for one-line blocks.

 > +if (NULL == cqp_request) {

It's more idiomatic to write "if (!cqp_request) {"
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Jeff Garzik]

Michael Buesch wrote:

On Wednesday 08 August 2007 18:59:08 Jeff Garzik wrote:

Michael Buesch wrote:

writel doesn't guarantee flushing either.
readl does.


Not quite -- there are multiple kinds of flushing.  You're thinking 
about flushing across PCI bridges, which is correct, but you also have 
CPU write posting and CPU write ordering and such.


Without taking all that into account, you might be tempted to think that 
__raw_readl() will perform all flushes necessary following a 
__raw_writel() -- but that would be incorrect.


So, kind of...
Better use writel(swab32(...
unless you like being shot into the foot.


Correct.

Jeff



___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] [GIT PULL] ~sashak/management.git

[Sasha Khapyorsky]

Hi Vlad,

Please pull recent ofed_1_2 branch of ~sashak/management.git.

In addition to already pulled:

commit 067fc2c8d91b405546d1d5629540880681c45df1
Author: Sasha Khapyorsky <[EMAIL PROTECTED]>

opensm: descrease default HoQLife value for leaf ports.

commit 2705d36f473ea80f571a91cde47e44d90d223b0f
Author: Yevgeny Kliteynik <[EMAIL PROTECTED]>

osm: default leaf vl_stall and head_of_queue_life counters


There are also:

commit 46bdba974ee2e1c8a64101effdb7358fd9060c8b
Author: Sasha Khapyorsky <[EMAIL PROTECTED]>

opensm: bump OpenSM version to 3.0.14

commit adb9138f45d13a7479dd7b839134ae50a7bbb152
Author: Sasha Khapyorsky <[EMAIL PROTECTED]>

opensm/ChangeLog: add recent changes description

Thanks,
Sasha
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: problems with ofed-1.2.c chelsio low level driver

[Michael S. Tsirkin]

> Quoting Steve Wise <[EMAIL PROTECTED]>:
> Subject: problems with ofed-1.2.c chelsio low level driver
> 
> Michael,
> 
> I've discovered that I was really testing the ofed-1.2 cxgb3.ko module 
> in my testing 1.2.c regression test.  Upon getting a complete 1.2.c 
> installed, I'm seeing crashes with the chelsio driver.  I think there 
> are some patches that went into the ofed-1.2 branch, that did not get 
> merged into ofed-1.2.c.   The ofed-1.2 branch works, but 1.2.c is broken.
>
> I'm investigating now, but we'll need to fix this before shipping 
> 1.2.c/1.2.5.

Sure, send 'em over.

-- 
MST
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] problems with ofed-1.2.c chelsio low level driver

[Steve Wise]

Michael,

I've discovered that I was really testing the ofed-1.2 cxgb3.ko module 
in my testing 1.2.c regression test.  Upon getting a complete 1.2.c 
installed, I'm seeing crashes with the chelsio driver.  I think there 
are some patches that went into the ofed-1.2 branch, that did not get 
merged into ofed-1.2.c.   The ofed-1.2 branch works, but 1.2.c is broken.


I'm investigating now, but we'll need to fix this before shipping 
1.2.c/1.2.5.



Steve.

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [ofa-general] RE: OFA website edits

[Jeff Becker]

Hi. I created most of the requested directory/owner pairs in
/var/www/openfabrics.org/downloads. I left out the various MPI
directories, figuring the appropriate web pages will be linked from
somewhere (possibly the downloads web page). I gave Stan Smith an
account. Stan, please contact me to get the account info.

I'm still working out how to do the dynamic web page stuff, but at
least people can start populating their directories.

Thanks.

-jeff

On 7/25/07, Arlin Davis <[EMAIL PROTECTED]> wrote:
>
> > I would like to propose adding project directories under
> > http://www.openfabrics.org/downloads/  where appropriate and give
> > maintainers access. For example:
> >
> Jeff,  please add the following directories with maintainer access as
> follow (or grant access at a maintainer group level):
>
> http://www.openfabrics.org/downloads/verbs (rdreier)
> http://www.openfabrics.org/downloads/rdmacm (shefty)
> http://www.openfabrics.org/downloads/dapl (ardavis)
> http://www.openfabrics.org/downloads/sdp (eitan)
> http://www.openfabrics.org/downloads/utils (eitan)
> http://www.openfabrics.org/downloads/management (sashak)
> http://www.openfabrics.org/downloads/OFED (vlad)
> http://www.openfabrics.org/downloads/archives (vlad)
> http://www.openfabrics.org/downloads/WinOF (ssmith)   (Stan Smith will
> need an account)
> http://www.openfabrics.org/downloads/hw/mthca (rdreir)
> http://www.openfabrics.org/downloads/hw/mlx4 (rdreir)
> http://www.openfabrics.org/downloads/hw/ehca (raisch)
> http://www.openfabrics.org/downloads/hw/ipath (ralphc)
> http://www.openfabrics.org/downloads/hw/cxgb3 (ralphc)
> http://www.openfabrics.org/downloads/mpi/mvapich (pasha)
> http://www.openfabrics.org/downloads/mpi/mvapich2 (rowland)
> http://www.openfabrics.org/downloads/mpi/openmpi (jsquyres)
>
> Let us know when these directories are created and the maintainers, who
> want to expose their packages via the webpage, will create a README that
> details the contents of the directory along with WEB_README that
> provides a short description for the webpage.
>
> Will this format allow you to auto configure the download webpage
> sufficiently? The idea is to only add links/descriptions to those
> project sub-directories with WEB_README files present.
>
> Please advise if something on the list is wrong or we missed a project.
>
> Thanks,
>
> -arlin
>
>
> ___
> general mailing list
> [EMAIL PROTECTED]
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 18:59:08 Jeff Garzik wrote:
> Michael Buesch wrote:
> > writel doesn't guarantee flushing either.
> > readl does.
> 
> 
> Not quite -- there are multiple kinds of flushing.  You're thinking 
> about flushing across PCI bridges, which is correct, but you also have 
> CPU write posting and CPU write ordering and such.
> 
> Without taking all that into account, you might be tempted to think that 
> __raw_readl() will perform all flushes necessary following a 
> __raw_writel() -- but that would be incorrect.

So, kind of...
Better use writel(swab32(...
unless you like being shot into the foot.

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] [PATCH 0/14] nes: NetEffect 10Gb RNIC Driver

[Roland Dreier]

 > git.openfabrics.org/~glenn/libnes.git
 > git.openfabrics.org/~glenn/ofed_1_2.git
 > git.openfabrics.org/~glenn/ofascripts.git
 > git.openfabrics.org/~glenn/ofed_1_2_scripts.git

these aren't actually git URLs.  prepending git:// seems to work though.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] [PATCH 4/5] libnes: OpenFabrics userspace verbs

[Roland Dreier]

 > +nesupd->udoorbell = mmap(NULL, page_size, PROT_WRITE | PROT_READ, 
 > MAP_SHARED,
 > +context->cmd_fd, nesupd->db_index * page_size);
 > +
 > +if (((void *)-1) == nesupd->udoorbell) {

probably cleaner to test against MAP_FAILED here.

 > +asm __volatile__("": : :"memory");

This is insufficient on architectures with weak memory ordering; you
probably want mb() from .
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] [PATCH 1/5] libnes: library init entry points

[Roland Dreier]

 > +ibv_driver_init;

This is unnecessary -- I realize you copied it from libmthca or
libmlx4 probably, but there it is a relic of older code.

I'll delete it from the libmthca and libmlx4 linker script to avoid
confusing anyone else.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Jeff Garzik]

Michael Buesch wrote:

writel doesn't guarantee flushing either.
readl does.



Not quite -- there are multiple kinds of flushing.  You're thinking 
about flushing across PCI bridges, which is correct, but you also have 
CPU write posting and CPU write ordering and such.


Without taking all that into account, you might be tempted to think that 
__raw_readl() will perform all flushes necessary following a 
__raw_writel() -- but that would be incorrect.


Jeff


___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[akepner]

On Wed, Aug 08, 2007 at 09:46:16AM -0700, Roland Dreier wrote:

> 
> Not mmiowb() -- that is for ordering between CPUs, eg on systems like
> Altix where PCI transactions might get reordered in the system fabric
> before reaching the PCI bus.
> 

Yes, that's right. This is a continual source of confusion. 
FWIW, Jesse Barnes documented mmiowb() in 
Documentation/DocBook/deviceiobook.tmpl 

-- 
Arthur

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Roland Dreier]

 > The barrier/ordering issue however might be a critical thing,
 > when using __raw_XXX. So one must always mmiowb() after such a write.

Not mmiowb() -- that is for ordering between CPUs, eg on systems like
Altix where PCI transactions might get reordered in the system fabric
before reaching the PCI bus.

You need a full wmb() to order between __raw_writel()s.

 - R.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 18:33:24 Jeff Garzik wrote:
> Michael Buesch wrote:
> > On Wednesday 08 August 2007 18:18:31 Roland Dreier wrote:
> >>  > But there are indeed a few cases that look wrong.
> >>
> >> yes...
> >>
> >>  > arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), 
> >> target);
> >>
> >> eg this almost certainly wants to be
> >>
> >>writel(swab32(val), target);
> >>
> >> or something equivalent like
> >>
> >>__raw_writel(cpu_to_be32(val), target);
> >>/* plus some suffficent memory ordering */
> >>
> >>  - R.
> >>
> >>
> > 
> > certainly, yes.
> > Most likely the __raw_writel variant is portable, but I am not
> > sure. Anybody sure?
> 
> Yes, it's portable.  You must however be aware of the guarantees that 
> writel() provides and __raw_writel() does not:  no barriers or flushes, 
> no endian conversions, no ordering constraints, ...  Probably a few more 
> details I'm forgetting too :)

writel doesn't guarantee flushing either.
readl does.
The barrier/ordering issue however might be a critical thing,
when using __raw_XXX. So one must always mmiowb() after such a write.

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Roland Dreier]

 > Most likely the __raw_writel variant is portable, but I am not
 > sure. Anybody sure?

Yes, it should be fine.  I use that construct in a few IB drivers.

 - R.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Jeff Garzik]

Michael Buesch wrote:

On Wednesday 08 August 2007 18:18:31 Roland Dreier wrote:

 > But there are indeed a few cases that look wrong.

yes...

 > arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), target);

eg this almost certainly wants to be

writel(swab32(val), target);

or something equivalent like

__raw_writel(cpu_to_be32(val), target);
/* plus some suffficent memory ordering */

 - R.




certainly, yes.
Most likely the __raw_writel variant is portable, but I am not
sure. Anybody sure?


Yes, it's portable.  You must however be aware of the guarantees that 
writel() provides and __raw_writel() does not:  no barriers or flushes, 
no endian conversions, no ordering constraints, ...  Probably a few more 
details I'm forgetting too :)


Jeff



___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 18:18:31 Roland Dreier wrote:
>  > But there are indeed a few cases that look wrong.
> 
> yes...
> 
>  > arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), target);
> 
> eg this almost certainly wants to be
> 
>   writel(swab32(val), target);
> 
> or something equivalent like
> 
>   __raw_writel(cpu_to_be32(val), target);
>   /* plus some suffficent memory ordering */
> 
>  - R.
> 
> 

certainly, yes.
Most likely the __raw_writel variant is portable, but I am not
sure. Anybody sure?

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Jeff Garzik]

Roland Dreier wrote:

 > But there are indeed a few cases that look wrong.

yes...

 > arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), target);

eg this almost certainly wants to be

writel(swab32(val), target);

or something equivalent like

__raw_writel(cpu_to_be32(val), target);
/* plus some suffficent memory ordering */


Precisely.  Some of those cases are "we know the underlying writel() 
swaps... we want to swap in this case anyway"


Jeff



___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Jeff Garzik]

Andi Kleen wrote:

Jeff Garzik <[EMAIL PROTECTED]> writes:

+   val, reg_index, addr, addr+4); */
+   writel(cpu_to_le32(reg_index), addr);
+   writel(cpu_to_le32(val),(u8 *)addr + 4);

wrong -- endian conversion macros not needed with writel()


Are you sure?


Yes.

read[bwl] and write[bwl] are always defined in terms of the 
little-endian PCI bus.  This has been true since my first days in the 
kernel ages (decade+) ago, when we had a long discussion about it with 
regards to framebuffer drivers.


If you want to skip barriers and endian conversions, __raw_write[bwl]() 
exists.


The rare exceptions are a few embedded arches that implemented writel() 
for a non-PCI bus.  Those cases need to be renamed to mybus_writel(), 
but at least they do not interfere with mainstream drivers and APIs.



I don't think that's true. e.g. powerpc writel 
doesn't convert endian


Incorrect -- read the code.  PPC most certainly does convert endian.

Ten years ago Linus said something along the lines of "writel() means 
PCI means little endian.  period."  ;-)


Jeff


___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Roland Dreier]

 > But there are indeed a few cases that look wrong.

yes...

 > arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), target);

eg this almost certainly wants to be

writel(swab32(val), target);

or something equivalent like

__raw_writel(cpu_to_be32(val), target);
/* plus some suffficent memory ordering */

 - R.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] RE: [PATCH 3/14] nes: connection manager routines

[Glenn Grundstrom]

This code is far from a TCP stack.  It's main purpose is to exchange
RDMA MPA request/response messages that are required by the iWarp specs
and therefore needed by our hardware.  All RNIC hardware vendors need
this MPA message exchange, including those already accepted into
kernel.org.  Do you have an alternative suggestion?

Thanks,
Glenn.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andi Kleen
Sent: Wednesday, August 08, 2007 7:41 AM
To: Glenn Grundstrom
Cc: [EMAIL PROTECTED]; ewg@lists.openfabrics.org; [EMAIL PROTECTED]
Subject: Re: [PATCH 3/14] nes: connection manager routines

[EMAIL PROTECTED] writes:

> NetEffect connection manager routines.

This seems more like a new TCP stack. I don't think such code is
appropiate, since Linux already has one.

-Andi
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] OFED 1.2.c-11 is available

[Tziporet Koren]

Erez Zilber wrote:

Tziporet,


Can you add the following as a known issue:


"open-iscsi over TCP does not support usage of data digest on RedHat EL/AS 4 
up3,
up4 & up5."

Note that this comment should be added to OFED release notes, not to
iSER release notes. This is not an iSER bug (it's a bug in the backport
for open-iscsi).

Thanks,
Erez

  

Will do in next build

tziporet

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] OFED 1.2.c status & plans

[Steve Wise]

Tziporet,

Can we make the change to 1.2.5?  This should probably include:

- change builds/connectx to builds/ofed-1.2.5 (or just add a link)
- change build names from 1.2.c to 1.2.5





Scott Weitzenkamp (sweitzen) wrote:
Cisco has been testing 1.2.c-10 IPoIB/SDP/MPI successfully on a 32-node 
cluster.  We are still working on tvflash, though.
 
Scott



*From:* [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] *On Behalf Of *Tziporet Koren
*Sent:* Tuesday, August 07, 2007 12:06 PM
*To:* EWG
*Cc:* OpenFabrics General
*Subject:* [ewg] OFED 1.2.c status & plans

Hi All,

I wish to update on OFED 1.2.c status and plans to synch everybody:

* OFED 1.2.c-11 is going out tomorrow
* This release should be the base for the GA release
* Need an approval from Steve (Chelsio) & Nam (IBM) that
  everything is in place from their perspective.
  Also please send me the release notes for ehca and cxgb3
* Need an approval from the companies that are testing this
  release that it can go to GA
   >From Mellanox perspective (mlx4 readiness) we are ready for GA.

I have one question: do we prefer to stay with the name 1.2.c or 1.2.5?

Thanks,
Tziporet




___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] RE: [PATCH 6/14] nes: hardware init

[Glenn Grundstrom]

Jeff,

Thanks for the input.  I'll take your suggestions into account for the
patch v2 posting.

Glenn.

-Original Message-
From: Jeff Garzik [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, August 07, 2007 8:58 PM
To: Glenn Grundstrom
Cc: [EMAIL PROTECTED]; ewg@lists.openfabrics.org; [EMAIL PROTECTED]
Subject: Re: [PATCH 6/14] nes: hardware init

[EMAIL PROTECTED] wrote:
> +struct nes_adapter *nes_init_adapter(struct nes_device *nesdev, u8
hw_rev) {
> + struct nes_adapter *nesadapter = NULL;
> + unsigned long num_pds;
> + u32 u32temp;
> + u32 port_count;
> + u16 max_rq_wrs;
> + u16 max_sq_wrs;
> + u32 max_mr;
> + u32 max_256pbl;
> + u32 max_4kpbl;
> + u32 max_qp;
> + u32 max_irrq;
> + u32 max_cq;
> + u32 hte_index_mask;
> + u32 adapter_size;
> + u32 arp_table_size;
> + u8  OneG_Mode;
> +
> + /* search the list of existing adapters */
> + list_for_each_entry(nesadapter, &nes_adapter_list, list) {
> + dprintk("Searching Adapter list for PCI devfn = 0x%X,"
> + " adapter PCI slot/bus = %u/%u, pci
devices PCI slot/bus = %u/%u, .\n",
> + nesdev->pcidev->devfn,
> + PCI_SLOT(nesadapter->devfn),
> + nesadapter->bus_number,
> + PCI_SLOT(nesdev->pcidev->devfn),
> + nesdev->pcidev->bus->number );
> + if ((PCI_SLOT(nesadapter->devfn) ==
PCI_SLOT(nesdev->pcidev->devfn)) &&
> + (nesadapter->bus_number ==
nesdev->pcidev->bus->number)) {
> + nesadapter->ref_count++;
> + return(nesadapter);

you don't need any of this PCI bus scanning at all.  Please convert to
normal PCI usage



> + /* no adapter found */
> + num_pds = pci_resource_len(nesdev->pcidev, BAR_1) / 4096;

see, this is why the BAR_1 define should go away -- it's actually define

to the value '2'


> + if (hw_rev != NE020_REV) {
> + dprintk("%s: NE020 driver detected unknown hardware
revision 0x%x\n",
> + __FUNCTION__, hw_rev);
> + return(NULL);
> + }

move this test to the top of the function


> + dprintk("%s:%u Determine Soft Reset, QP_control=0x%x, CPU0=0x%x,
CPU1=0x%x, CPU2=0x%x\n",
> + __FUNCTION__, __LINE__,
> + nes_read_indexed(nesdev, NES_IDX_QP_CONTROL +
PCI_FUNC(nesdev->pcidev->devfn) * 8),
> + nes_read_indexed(nesdev,
NES_IDX_INT_CPU_STATUS),
> + nes_read_indexed(nesdev, 0x00A4),
> + nes_read_indexed(nesdev, 0x00A8));
> +
> + dprintk("%s: Reset and init NE020\n", __FUNCTION__);
> + if ((port_count = nes_reset_adapter_ne020(nesdev,
&OneG_Mode)) == 0) {
> + return(NULL);
> + }
> + if (nes_init_serdes(nesdev, port_count)) {
> + return(NULL);
> + }

kill braces


> + nes_init_csr_ne020(nesdev, hw_rev, port_count);
> +
> + /* Setup and enable the periodic timer */
> + nesdev->et_rx_coalesce_usecs_irq = interrupt_mod_interval;
> + if (nesdev->et_rx_coalesce_usecs_irq) {
> + nes_write32(nesdev->regs+NES_PERIODIC_CONTROL,
0x8000 |
> + ((u32)(nesdev->et_rx_coalesce_usecs_irq
* 8)));
> + } else {
> + nes_write32(nesdev->regs+NES_PERIODIC_CONTROL,
0x);
> + }
> +
> + max_qp = nes_read_indexed(nesdev, NES_IDX_QP_CTX_SIZE);
> + dprintk("%s: QP_CTX_SIZE=%u\n", __FUNCTION__, max_qp);
> +
> + u32temp = nes_read_indexed(nesdev,
NES_IDX_QUAD_HASH_TABLE_SIZE);
> + if (max_qp > ((u32)1 << (u32temp & 0x001f))) {
> + dprintk("Reducing Max QPs to %u due to hash table size =
0x%08X\n",
> + max_qp, u32temp);
> + max_qp = (u32)1 << (u32temp & 0x001f);
> + }
> +
> + hte_index_mask = ((u32)1 << ((u32temp & 0x001f)+1))-1;
> + dprintk("Max QP = %u, hte_index_mask = 0x%08X.\n", max_qp,
hte_index_mask);
> +
> + u32temp = nes_read_indexed(nesdev, NES_IDX_IRRQ_COUNT);
> +
> + max_irrq = 1 << (u32temp & 0x001f);
> +
> + if (max_qp > max_irrq) {
> + max_qp = max_irrq;
> + dprintk("Reducing Max QPs to %u due to Available
Q1s.\n", max_qp);
> + }
> +
> + /* there should be no reason to allocate more pds than qps */
> + if (num_pds > max_qp)
> + num_pds = max_qp;
> +
> + u32temp = nes_read_indexed(nesdev, NES_IDX_MRT_SIZE);
> + max_mr = (u32)8192 << (u32temp & 0x7);
> +
> + u32temp = nes_read_indexed(nesdev, NES_IDX_PBL_REGION_SIZE);
> + max_256pbl = (u32)1 << (u32temp & 0x001f);
> + max_4kpbl = (u32)1 << ((u32temp >> 16) & 0x001f);
> + max_cq = nes_read_indexed(nesdev, NES_IDX_CQ_CTX_SIZE);
> +
> + u32temp = nes_read_indexed(nesdev, NES

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 15:48:25 Michael Buesch wrote:
> > However if that's true then there are quite some drivers wrong:
> > 
> > % grep -r write[bwl]\(cpu_to *   | wc -l
> > 57
> 
> Yeah, seems so.

Most of them seem to be
 __raw_writew(cpu_toXX(...

which I think might be valid.
But there are indeed a few cases that look wrong.

arch/mips/pci/ops-bonito64.c:   writel(cpu_to_le32(*data), addrp);
arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), target);
arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), target);
arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val32), 
target);
arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val), target);
arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val32), target);
arch/x86_64/kernel/pci-calgary.c:   writel(cpu_to_be32(val32), target);
drivers/atm/fore200e.c:writel(cpu_to_le32(val), addr);
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[2]), setup_frm); 
setup_frm += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[1]), setup_frm); 
setup_frm += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[0]), setup_frm); 
setup_frm += 8;
drivers/net/starfire.c: writew(cpu_to_be16(i), 
filter_addr);
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[2]), 
filter_addr); filter_addr += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[1]), 
filter_addr); filter_addr += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[0]), 
filter_addr); filter_addr += 8;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[0]), 
filter_addr); filter_addr += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[1]), 
filter_addr); filter_addr += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[2]), 
filter_addr); filter_addr += 8;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[0]), 
filter_addr); filter_addr += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[1]), 
filter_addr); filter_addr += 4;
drivers/net/starfire.c: writew(cpu_to_be16(eaddrs[2]), 
filter_addr); filter_addr += 8;
drivers/net/hamachi.c:  writel(cpu_to_le64(hmp->rx_ring_dma), ioaddr + RxPtr);
drivers/net/hamachi.c:  writel(cpu_to_le64(hmp->rx_ring_dma) >> 32, ioaddr + 
RxPtr + 4);
drivers/net/hamachi.c:  writel(cpu_to_le64(hmp->tx_ring_dma), ioaddr + TxPtr);
drivers/net/hamachi.c:  writel(cpu_to_le64(hmp->tx_ring_dma) >> 32, ioaddr + 
TxPtr + 4);
drivers/net/hamachi.c:  writel(cpu_to_le32(hmp->rx_ring_dma), ioaddr + RxPtr);
drivers/net/hamachi.c:  writel(cpu_to_le32(hmp->tx_ring_dma), ioaddr + TxPtr);
drivers/net/tokenring/lanstreamer.c:
writel(cpu_to_le32(pci_map_single(streamer_priv->pci_dev, rx_ring, 512, 
PCI_DMA_FROMDEVICE)),
drivers/net/tokenring/lanstreamer.c:
writel(cpu_to_le32(pci_map_single(streamer_priv->pci_dev, 
&streamer_priv->streamer_rx_ring[0],
drivers/net/tokenring/lanstreamer.c:
writel(cpu_to_le32(pci_map_single(streamer_priv->pci_dev, 
&streamer_priv->streamer_rx_ring[STREAMER_RX_RING_SIZE - 1],
drivers/net/tokenring/lanstreamer.c:
writel(cpu_to_le32(pci_map_single(streamer_priv->pci_dev, 
drivers/net/tokenring/lanstreamer.c:
writel(cpu_to_le32(pci_map_single(streamer_priv->pci_dev, 
drivers/net/tokenring/lanstreamer.c:
writel(cpu_to_le32(pci_map_single(streamer_priv->pci_dev, 
drivers/net/via-velocity.c: writel(cpu_to_le32(vptr->rd_pool_dma), 
®s->RDBaseLo);
drivers/net/via-velocity.c: 
writel(cpu_to_le32(vptr->td_pool_dma[i]), &(regs->TDBaseLo[i]));
drivers/scsi/nsp32_io.h:writew(cpu_to_le16(val), ptr);
drivers/scsi/nsp32_io.h:writel(cpu_to_le32(val), ptr);
drivers/scsi/nsp32_io.h:writew(cpu_to_le16(val), data_ptr );
drivers/block/umem.c:   writel(cpu_to_le32((page->page_dma+offset)&0x),
drivers/block/umem.c:   writel(cpu_to_le32(((u64)page->page_dma)>>32),
drivers/block/umem.c:   writel(cpu_to_le32(DMASCR_GO | DMASCR_CHAIN_EN | 
pci_cmds),
drivers/block/umem.c:   
writel(cpu_to_le32(DMASCR_DMA_COMPLETE|DMASCR_CHAIN_COMPLETE),

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 15:38:50 Andi Kleen wrote:
> On Wed, Aug 08, 2007 at 03:28:33PM +0200, Michael Buesch wrote:
> > On Wednesday 08 August 2007 15:08:28 Andi Kleen wrote:
> > > On Wed, Aug 08, 2007 at 03:02:35PM +0200, Michael Buesch wrote:
> > > > On Wednesday 08 August 2007 14:55:11 Andi Kleen wrote:
> > > > > On Wed, Aug 08, 2007 at 01:50:35PM +0200, Michael Buesch wrote:
> > > > > > On Wednesday 08 August 2007 14:38:10 Andi Kleen wrote:
> > > > > > > Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > > > > > > > + val, reg_index, addr, addr+4); */
> > > > > > > > > + writel(cpu_to_le32(reg_index), addr);
> > > > > > > > > + writel(cpu_to_le32(val),(u8 *)addr + 4);
> > > > > > > > 
> > > > > > > > wrong -- endian conversion macros not needed with writel()
> > 
> > > > Fact is that we do _not_ need cpu_to_le32 with writel.
> > > 
> > > We do on a big endian platform if the register is LE. I assume that's the 
> > > case 
> > > on this hardware.
> > 
> > That is not true.
> > writeX does automatically convert to bus-endian.
> > Which, in case of the PCI bus, is little endian.
> > So if your register is LE (which it is most likely), you don't
> > need any conversion. If your register is BE (which I very much doubt),
> > then you need swab32().
> > In _no_ case you need cpu_to_xx().
> 
> Hmm, I checked a couple of BE architectures and none seem to convert 
> explicitely.

That depends on the arch.
Look at this from ppc, for example:

184 static inline void writel(__u32 b, volatile void __iomem *addr)
185 {
186 out_le32(addr, b);
187 }

out_le32 writes a little endian value. So it converts it.
Also note that b is __u32. Sparse would complain, if someone used cpu_to_xx
on it.

> But ok it's possible that their PCI bridges convert. I'll take your
> word for it although it sounds somewhat dubious.
>
> However if that's true then there are quite some drivers wrong:
> 
> % grep -r write[bwl]\(cpu_to *   | wc -l
> 57

Yeah, seems so.
Here comes an example (with 16bit values)

Little endian register
LittleEndian arch   BigEngian arch
value   0xBBAA  0xAABB
writew  0xBBAA  0xAABB
on bus  0xBBAA  0xBBAA
on dev  0xBBAA  0xBBAA

Big endian register
LittleEndian arch   BigEngian arch
value   0xBBAA  0xAABB
swab16  0xAABB  0xBBAA
writew  0xAABB  0xBBAA
on bus  0xAABB  0xAABB
on dev  0xAABB  0xAABB

I hope I got it right. :)
Same result on every arch.

Most hardware has little endian registers. Some can switch
endianess based on some bit, too. So in case of a BE register that
bit has to be flipped, or if not possible swabX() has to be used.

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] OFED 1.2.c-11 is available

[Erez Zilber]

Tziporet,


Can you add the following as a known issue:


"open-iscsi over TCP does not support usage of data digest on RedHat EL/AS 4 
up3,
up4 & up5."

Note that this comment should be added to OFED release notes, not to
iSER release notes. This is not an iSER bug (it's a bug in the backport
for open-iscsi).

Thanks,
Erez


Tziporet Koren wrote:

> Hi All,
>
> OFED 1.2.c-11 is available now on the OFA server under:
> _http://www.openfabrics.org/builds/connectx/release/_
> Note: This release supports FW 2.1.000 that is available on Mellanox
> web site for download
>   and FW 2.2.0 that will be released soon
>
> Supported Platforms and Operating Systems
> =
> o CPU architectures:
> - x86_64
> - x86
> - ppc64
> - ia64
>
> o Linux Operating Systems:
> - RedHat EL4 up3: 2.6.9-34.ELsmp
> - RedHat EL4 up4: 2.6.9-42.ELsmp
> - RedHat EL4 up5: 2.6.9-55.ELsmp
> - RedHat EL5: 2.6.18-8.el5
> - SLES10: 2.6.16.21-0.8-smp
> - kernel.org: 2.6.20.x and 2.6.22.x
> - SLES10 SP1: 2.6.16.46-0.12-smp (partially tested)
>
> Main changes from OFED 1.2.c-10:
> 
> - Merged all 1.2 branch patches to 1.2.c
> - mad: Fix address handle leak in mad_rmpp
> - mlx4
> - Added support for interrupt moderation for create cq to
> support ipoib.
> - Display the following device information via sysfs:
> board_id, fw_ver,
>   hw_rev and hca_type. As a result ibstat presenting all entries.
> - Fixed destroy special QPs.
> - Fixed the work completion opcode for RDMA_READ operation
> - opensm:
> - Sets the time a packet can live in the Head of Queue of a
> port that
>   drives a CA port to approximately 1 second.
> - Number of dropped packets due to HOQ that will stall the CA
> was increased
>  to 7 (instead of 1).
> - IPoIB:
> - Add interrupt moderation support for IPoIB UD
> - NAPI is available using a module parameter
> - Fixed a leak in ipoib_transport_dev_init
> - Fixed kernel oops in IPoIB download
> - New MVAPICH package (mvapich-0.9.9-1451):
> - mpirun_rsh fixes:
>   * Check that we do not overflow command line size
>   * better handling for accept
> - Fix for memory allocations whose size is bigger than 2^31-1
> - Fix for multiple communicator problem
> - Fix for MPI_Finalize segmentation fault
> - RDS:
> - rds_poll() always returns POLLOUT.  If you try to send and get
>   EWOULDBLOCK because the remote receiver is congested then
> you don't get
>   to wait for POLLOUT to be raised before sending again --
> it's always
>   raised.  This is worked around by implementing some kind of
> exponential
>   back-off while retrying the send.  This can be done by
> increasing the
>   timeout given to poll().  At each poll() timeout expiry the
> send is
>   tried again.
> - Set max send and receive scatter/gather list size to 2 in
> the QP attributes.
>   The second sge used for RDS header.
> - Added rs_poll_flag to rds_socket structure to indicate the
> back pressure
>   status and send space availability.
> - Added MODULE_VERSION.
> - ehca:
> - Added support for RHEL4 up5
>
>
> Tested ULPs
> ===
> The following ULPs are tested:
> o OpenSM
> o MPI (MVAPICH and Open MPI)
> o IPoIB UD and CM
> o SDP
> o SRP
> o iSER
> o RDS
>
> ConnectX specific known issues and limitations
> ==
> - The following features are not supported:
>   o Resize CQ
>   o APM
>   o SQD
> - To load the driver on machines with 64KB default page size UAR bar
> must be
>   enlarged. 64KB page size is the default of PPC with RHEL5 and
> Itanium with
>   64KB page size enabled.
>   Perform the following three steps:
>   1. Add the following line in the firmware configuration (INI) file
> under the
>  [HCA] section:
>log2_uar_bar_megabytes = 5
>   2. Burn a modified firmware image with the changed INI file
>   3. Reboot the system
>
>
> Tziporet Koren
> Software Director
> Mellanox Technologies
> mailto: [EMAIL PROTECTED] 
> Tel +972-4-9097200, ext 380
>
> 
>
> ___
> ewg mailing list
> ewg@lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Andi Kleen]

On Wed, Aug 08, 2007 at 03:28:33PM +0200, Michael Buesch wrote:
> On Wednesday 08 August 2007 15:08:28 Andi Kleen wrote:
> > On Wed, Aug 08, 2007 at 03:02:35PM +0200, Michael Buesch wrote:
> > > On Wednesday 08 August 2007 14:55:11 Andi Kleen wrote:
> > > > On Wed, Aug 08, 2007 at 01:50:35PM +0200, Michael Buesch wrote:
> > > > > On Wednesday 08 August 2007 14:38:10 Andi Kleen wrote:
> > > > > > Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > > > > > > +   val, reg_index, addr, addr+4); */
> > > > > > > > +   writel(cpu_to_le32(reg_index), addr);
> > > > > > > > +   writel(cpu_to_le32(val),(u8 *)addr + 4);
> > > > > > > 
> > > > > > > wrong -- endian conversion macros not needed with writel()
> 
> > > Fact is that we do _not_ need cpu_to_le32 with writel.
> > 
> > We do on a big endian platform if the register is LE. I assume that's the 
> > case 
> > on this hardware.
> 
> That is not true.
> writeX does automatically convert to bus-endian.
> Which, in case of the PCI bus, is little endian.
> So if your register is LE (which it is most likely), you don't
> need any conversion. If your register is BE (which I very much doubt),
> then you need swab32().
> In _no_ case you need cpu_to_xx().

Hmm, I checked a couple of BE architectures and none seem to convert 
explicitely.
But ok it's possible that their PCI bridges convert. I'll take your
word for it although it sounds somewhat dubious.

However if that's true then there are quite some drivers wrong:

% grep -r write[bwl]\(cpu_to *   | wc -l
57

-Andi

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 15:08:28 Andi Kleen wrote:
> On Wed, Aug 08, 2007 at 03:02:35PM +0200, Michael Buesch wrote:
> > On Wednesday 08 August 2007 14:55:11 Andi Kleen wrote:
> > > On Wed, Aug 08, 2007 at 01:50:35PM +0200, Michael Buesch wrote:
> > > > On Wednesday 08 August 2007 14:38:10 Andi Kleen wrote:
> > > > > Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > > > > > + val, reg_index, addr, addr+4); */
> > > > > > > + writel(cpu_to_le32(reg_index), addr);
> > > > > > > + writel(cpu_to_le32(val),(u8 *)addr + 4);
> > > > > > 
> > > > > > wrong -- endian conversion macros not needed with writel()

> > Fact is that we do _not_ need cpu_to_le32 with writel.
> 
> We do on a big endian platform if the register is LE. I assume that's the 
> case 
> on this hardware.

That is not true.
writeX does automatically convert to bus-endian.
Which, in case of the PCI bus, is little endian.
So if your register is LE (which it is most likely), you don't
need any conversion. If your register is BE (which I very much doubt),
then you need swab32().
In _no_ case you need cpu_to_xx().

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Andi Kleen]

On Wed, Aug 08, 2007 at 03:02:35PM +0200, Michael Buesch wrote:
> On Wednesday 08 August 2007 14:55:11 Andi Kleen wrote:
> > On Wed, Aug 08, 2007 at 01:50:35PM +0200, Michael Buesch wrote:
> > > On Wednesday 08 August 2007 14:38:10 Andi Kleen wrote:
> > > > Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > > > > +   val, reg_index, addr, addr+4); */
> > > > > > +   writel(cpu_to_le32(reg_index), addr);
> > > > > > +   writel(cpu_to_le32(val),(u8 *)addr + 4);
> > > > > 
> > > > > wrong -- endian conversion macros not needed with writel()
> > > > 
> > > > Are you sure? I don't think that's true. e.g. powerpc writel 
> > > > doesn't convert endian
> > > 
> > > Andi, you are wrong.
> > > writeX take CPU endian args.
> > 
> > That is what I wrote.
> 
> Uhm, so
> Why did you complain to Jeff?

Because Jeff wrote the opposite.

> I'm a little bit confused now :)
> 
> Fact is that we do _not_ need cpu_to_le32 with writel.

We do on a big endian platform if the register is LE. I assume that's the case 
on this hardware.

-Andi
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 14:55:11 Andi Kleen wrote:
> On Wed, Aug 08, 2007 at 01:50:35PM +0200, Michael Buesch wrote:
> > On Wednesday 08 August 2007 14:38:10 Andi Kleen wrote:
> > > Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > > > + val, reg_index, addr, addr+4); */
> > > > > + writel(cpu_to_le32(reg_index), addr);
> > > > > + writel(cpu_to_le32(val),(u8 *)addr + 4);
> > > > 
> > > > wrong -- endian conversion macros not needed with writel()
> > > 
> > > Are you sure? I don't think that's true. e.g. powerpc writel 
> > > doesn't convert endian
> > 
> > Andi, you are wrong.
> > writeX take CPU endian args.
> 
> That is what I wrote.

Uhm, so
Why did you complain to Jeff?
I'm a little bit confused now :)

Fact is that we do _not_ need cpu_to_le32 with writel.

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] OFED 1.2.c-11 is available

[Tziporet Koren]

Hi All,

OFED 1.2.c-11 is available now on the OFA server under:
http://www.openfabrics.org/builds/connectx/release/
Note: This release supports FW 2.1.000 that is available on Mellanox web
site for download 
  and FW 2.2.0 that will be released soon

Supported Platforms and Operating Systems
=
o CPU architectures:
- x86_64
- x86
- ppc64
- ia64

o Linux Operating Systems:
- RedHat EL4 up3: 2.6.9-34.ELsmp
- RedHat EL4 up4: 2.6.9-42.ELsmp
- RedHat EL4 up5: 2.6.9-55.ELsmp
- RedHat EL5: 2.6.18-8.el5
- SLES10: 2.6.16.21-0.8-smp
- kernel.org: 2.6.20.x and 2.6.22.x
- SLES10 SP1: 2.6.16.46-0.12-smp (partially tested)

Main changes from OFED 1.2.c-10:

- Merged all 1.2 branch patches to 1.2.c
- mad: Fix address handle leak in mad_rmpp
- mlx4
- Added support for interrupt moderation for create cq to
support ipoib.
- Display the following device information via sysfs: board_id,
fw_ver,
  hw_rev and hca_type. As a result ibstat presenting all
entries.
- Fixed destroy special QPs.
- Fixed the work completion opcode for RDMA_READ operation
- opensm: 
- Sets the time a packet can live in the Head of Queue of a port
that
  drives a CA port to approximately 1 second.
- Number of dropped packets due to HOQ that will stall the CA
was increased 
 to 7 (instead of 1).
- IPoIB:
- Add interrupt moderation support for IPoIB UD
- NAPI is available using a module parameter
- Fixed a leak in ipoib_transport_dev_init
- Fixed kernel oops in IPoIB download
- New MVAPICH package (mvapich-0.9.9-1451):
- mpirun_rsh fixes:
  * Check that we do not overflow command line size
  * better handling for accept
- Fix for memory allocations whose size is bigger than 2^31-1
- Fix for multiple communicator problem
- Fix for MPI_Finalize segmentation fault
- RDS:
- rds_poll() always returns POLLOUT.  If you try to send and get
  EWOULDBLOCK because the remote receiver is congested then you
don't get
  to wait for POLLOUT to be raised before sending again -- it's
always
  raised.  This is worked around by implementing some kind of
exponential
  back-off while retrying the send.  This can be done by
increasing the
  timeout given to poll().  At each poll() timeout expiry the
send is
  tried again.
- Set max send and receive scatter/gather list size to 2 in the
QP attributes.
  The second sge used for RDS header.
- Added rs_poll_flag to rds_socket structure to indicate the
back pressure
  status and send space availability.
- Added MODULE_VERSION.
- ehca:
- Added support for RHEL4 up5


Tested ULPs
===
The following ULPs are tested:
o OpenSM
o MPI (MVAPICH and Open MPI)
o IPoIB UD and CM
o SDP
o SRP
o iSER
o RDS

ConnectX specific known issues and limitations
==
- The following features are not supported:
  o Resize CQ
  o APM
  o SQD
- To load the driver on machines with 64KB default page size UAR bar
must be
  enlarged. 64KB page size is the default of PPC with RHEL5 and Itanium
with
  64KB page size enabled.
  Perform the following three steps:
  1. Add the following line in the firmware configuration (INI) file
under the
 [HCA] section:
   log2_uar_bar_megabytes = 5
  2. Burn a modified firmware image with the changed INI file
  3. Reboot the system


Tziporet Koren
Software Director
Mellanox Technologies
mailto: [EMAIL PROTECTED]
Tel +972-4-9097200, ext 380

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Andi Kleen]

On Wed, Aug 08, 2007 at 01:50:35PM +0200, Michael Buesch wrote:
> On Wednesday 08 August 2007 14:38:10 Andi Kleen wrote:
> > Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > > +   val, reg_index, addr, addr+4); */
> > > > +   writel(cpu_to_le32(reg_index), addr);
> > > > +   writel(cpu_to_le32(val),(u8 *)addr + 4);
> > > 
> > > wrong -- endian conversion macros not needed with writel()
> > 
> > Are you sure? I don't think that's true. e.g. powerpc writel 
> > doesn't convert endian
> 
> Andi, you are wrong.
> writeX take CPU endian args.

That is what I wrote.

-Andi
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] [PATCH] IB/core: Ignore membership bit in ib_find_pkey()

[Moni Shoua]

Vlad,
This patch is already merged in upstream kernel.
Please add it also to kernel patches of OFED-1.2.5
thanks
   MoniS

--

ib_find_pkey() is used as a replacement for ib_find_cached_pkey(), and
the original function ignored the membership bit when searching for a
P_Key, so ib_find_pkey() should ignore the bit too.

In particular, IPoIB turns on the P_Key membership bit of limited
membership P_Keys when creating a child interface and looks for the
full membership P_key.  This broke if a port was a partial member of a
partition when IPoIB switched from ib_find_cached_pkey() to
ib_find_pkey(), and this change fixes things again.

Signed-off-by: Moni Shoua <[EMAIL PROTECTED]>
Signed-off-by: Roland Dreier <[EMAIL PROTECTED]>
---

diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 3ada17c..2506c43 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -702,7 +702,7 @@ int ib_find_pkey(struct ib_device *device,
if (ret)
return ret;
 
-   if (pkey == tmp_pkey) {
+   if ((pkey & 0x7fff) == (tmp_pkey & 0x7fff)) {
*index = i;
return 0;
}

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Michael Buesch]

On Wednesday 08 August 2007 14:38:10 Andi Kleen wrote:
> Jeff Garzik <[EMAIL PROTECTED]> writes:
> > > + val, reg_index, addr, addr+4); */
> > > + writel(cpu_to_le32(reg_index), addr);
> > > + writel(cpu_to_le32(val),(u8 *)addr + 4);
> > 
> > wrong -- endian conversion macros not needed with writel()
> 
> Are you sure? I don't think that's true. e.g. powerpc writel 
> doesn't convert endian

Andi, you are wrong.
writeX take CPU endian args.

-- 
Greetings Michael.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 3/14] nes: connection manager routines

[Andi Kleen]

[EMAIL PROTECTED] writes:

> NetEffect connection manager routines.

This seems more like a new TCP stack. I don't think such code
is appropiate, since Linux already has one.

-Andi
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] Re: [PATCH 2/14] nes: device structures and defines

[Andi Kleen]

Jeff Garzik <[EMAIL PROTECTED]> writes:
> > +   val, reg_index, addr, addr+4); */
> > +   writel(cpu_to_le32(reg_index), addr);
> > +   writel(cpu_to_le32(val),(u8 *)addr + 4);
> 
> wrong -- endian conversion macros not needed with writel()

Are you sure? I don't think that's true. e.g. powerpc writel 
doesn't convert endian

-Andi

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] OFED 1.2.c status & plans

[Tziporet Koren]

Scott Weitzenkamp (sweitzen) wrote:
Cisco has been testing 1.2.c-10 IPoIB/SDP/MPI successfully on a 
32-node cluster.  We are still working on tvflash, though.
 


We can add tvflash at beginning of next week

Tziporet

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

[ewg] ignore membership bit when looking for a P_key

[Michael S. Tsirkin]

Is this patch relevant for 1.2.5?
If yes pls feel free to send it to Vlad.

-- 
MST
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg