Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jul 20, 2010 at 9:49 PM, Isaku Yamahata wrote: > Added Cc: seab...@seabios.org > > On Wed, Jul 21, 2010 at 06:31:01AM +0300, Michael S. Tsirkin wrote: >> On Tue, Jul 20, 2010 at 06:52:23PM +0900, Isaku Yamahata wrote: >> > On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote: >> > > On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata >> > > wrote: >> > > > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote: >> > > >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata >> > > >> wrote: >> > > >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote: >> > > >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks >> > > >> >> >> > like it also >> > > >> >> >> > thinks the second half of the BAR is an I/O region instead of >> > > >> >> >> > memory (hence >> > > >> >> >> > the c200, that's part of the pci portio region. >> > > >> >> > >> > > >> >> > I've sent the patches to address it. But they haven't been >> > > >> >> > merged yet. >> > > >> >> > seabios doesn't map BARs beyond 4GB. >> > > >> >> > If bar is mapped beyond 4GB, guest BIOS does it. >> > > >> >> >> > > >> >> Have those patches been merged yet? >> > > >> > >> > > >> > They have been merged into seabios upstream now. >> > > >> > qemu seabios fork hasn't pulled for a while, though. >> > > >> > >> > > >> > >> > > >> >> > To see how seabios works, it would help to increase >> > > >> >> > CONFIG_DEBUG_LEVEL >> > > >> >> > in config.h of seabios >> > > >> >> >> > > >> >> Where does the output from seabios end up? ?Inside dmesg? >> > > >> > >> > > >> > It outputs them to the serial console which qemu emulates. >> > > >> > seabios is out of kernel control, so dmesg doesn't show it. >> > > >> > >> > > >> > >> > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) >> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) >> > > >> >> >> pci_read_config: (val) 0x <- 0x1c (addr) >> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) >> > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) >> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) >> > > >> >> > >> > > >> >> > seabios BAR3. Not sure how it is mapped from this >> > > >> >> > message. >> > > >> >> >> > > >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 >> > > >> >> and >> > > >> >> BAR3 to store all 64-bits? >> > > >> > >> > > >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature. >> > > >> > -- >> > > >> > yamahata >> > > >> > >> > > >> > >> > > >> >> > > >> With the latest seabios git passed via -bios, I no longer see the >> > > >> 48-bit address, but instead a 32-bit address and then >> > > >> . ?This guest has 1gb of RAM so the address isn't be >> > > >> mapped beyond 4g. >> > > > >> > > > Can I see the debug log like before? >> > > > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.) >> > > >> > > Here's the dump from SeaBIOS in the region related to the PCI devices. >> > > The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit. >> > > >> > > PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8 >> > > region 0: 0xf000 >> > > region 1: 0xf200 >> > > region 6: 0xf201 >> > > PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000 >> > > region 0: 0xc020 >> > > region 1: 0xf202 >> > > region 6: 0xf203 >> > > PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110 >> > > region 0: 0xf204 >> > > region 1: 0xf2041000 >> > > region 2: 0x >> > >> > Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110) >> > the BAR in quistion? >> > The value 0 seems odd. Probably BAR address calculation overflowed. >> > Currently seabios doesn't check overflow. I attached the patch. >> > >> > >> > > > Do you know who sets the BAR to ? >> > > >> > > Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM' >> > > lines are from the map function passed to pci_register_bar(). It >> > > looks like SeaBIOS sets the address to 0 and then the potentially >> > > useful e000 address gets mangled into 00. >> > >> > There is something wrong with the debug message of write case, I suppose. >> > All written value are 0, but the resulted effect doesn't seems so. >> > >> > > >> > > IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = >> > > 536870912 >> > > >> > > ...snip... >> > > >> > > pci_read_config: (val) 0x4 <- 0x18 (addr) >> > > pci_write_config: (val) 0x0 -> 0x18 (addr) >> > > IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size = >> > > 2000 >> > >> > If 0 is written to 0x18, the bar address should be 0, but it says e000. >> > >> > > pci_read_config: (val) 0xe004 <- 0x18 (addr) >> > >> > The read value isn't 0. and so on... >> > >> > > pci_write_config: (val) 0x0 -> 0x18 (addr) >> > > pci_read_config: (val) 0x0 <- 0x1c (addr) >> > > pci_write_config: (val) 0x0 -> 0x1c (addr) >> > > IVSHMEM: guest pci addr = , guest h/w ad
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Aug 24, 2010 at 10:52:36AM -0600, Cam Macdonell wrote:
> Hi, 64-bit BARs still do not seem to be working.
>
> When using the latest seabios the guest does not hit a "BUG:"
> statement, but booting still fails
>
> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
> divide error: [#1] SMP
> last sysfs file:
> CPU 0
> Modules linked in:
>
> Pid: 1, comm: swapper Not tainted 2.6.35+ #299 /Bochs
> RIP: 0010:[] [] hpet_alloc+0x12c/0x35b
> RSP: 0018:88007d7b3d80 EFLAGS: 00010246
> RAX: 00038d7ea4c68000 RBX: 88007d062cc0 RCX:
> RDX: RSI: RDI: 817bb9b0
> RBP: 88007d7b3dc0 R08: 80d0 R09: c900
> R10: 88007d72b5a0 R11: R12: 88007d7b3dd0
> R13: c900 R14: R15: 817a41c3
> FS: () GS:88000200() knlGS:
> CS: 0010 DS: ES: CR0: 8005003b
> CR2: CR3: 01a42000 CR4: 06f0
> DR0: DR1: DR2:
> DR3: DR6: 0ff0 DR7: 0400
> Process swapper (pid: 1, threadinfo 88007d7b2000, task 88007d7b8000)
> Stack:
> 88007f43ab90 88007f43ab90 81ca6174 81b1f5e1
> <0> 0100 0100
> <0> 88007d7b3e80 810294ea fed0 c900
> Call Trace:
> [] ? hpet_late_init+0x0/0xea
> [] hpet_reserve_platform_timers+0x10b/0x115
> [] ? hpet_late_init+0x0/0xea
> [] hpet_late_init+0x6b/0xea
> [] ? hpet_late_init+0x0/0xea
> [] do_one_initcall+0x5e/0x159
> [] kernel_init+0x19a/0x228
> [] kernel_thread_helper+0x4/0x10
> [] ? kernel_init+0x0/0x228
> [] ? kernel_thread_helper+0x0/0x10
> Code: 89 1d ca b2 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 c3 41 7a 81 48
> 8d 04 02 4c 89 f2 48 c7 c7 b0 b9 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
> f7 f1 83 7b 30 01 48 c7 c1 86 1c 7d 81 49 0f 46 cf 48 89 43
> RIP [] hpet_alloc+0x12c/0x35b
> RSP
> ---[ end trace a7919e7f17c0a725 ]---
> Kernel panic - not syncing: Attempted to kill init!
> Pid: 1, comm: swapper Tainted: G D 2.6.35+ #299
> Call Trace:
> [] panic+0x8b/0x10b
> [] ? exit_ptrace+0x38/0x121
> [] do_exit+0x7a/0x722
> [] ? spin_unlock_irqrestore+0xe/0x10
> [] ? kmsg_dump+0x12b/0x145
> [] oops_end+0xbf/0xc7
> [] die+0x5a/0x63
> [] do_trap+0x121/0x130
> [] do_divide_error+0x96/0x9f
> [] ? hpet_alloc+0x12c/0x35b
> [] ? radix_tree_preload+0x34/0x88
> [] divide_error+0x1b/0x20
> [] ? hpet_alloc+0x12c/0x35b
> [] ? hpet_late_init+0x0/0xea
> [] hpet_reserve_platform_timers+0x10b/0x115
> [] ? hpet_late_init+0x0/0xea
> [] hpet_late_init+0x6b/0xea
> [] ? hpet_late_init+0x0/0xea
> [] do_one_initcall+0x5e/0x159
> [] kernel_init+0x19a/0x228
> [] kernel_thread_helper+0x4/0x10
> [] ? kernel_init+0x0/0x228
> [] ? kernel_thread_helper+0x0/0x10
>
> seabios output for the device:
>
> PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
> region 0: 0xf102
> region 2: 0x
> init smm
>
> Running the latest seabios, the debug output only remaps the BAR
> twice, once with a potentially correct address of e
>
> pci_read_config: (val) 0xe004 <- 0x18 (addr)
The upstream seabios lacks overflow check at the moment.
I haven't found time to address PMM yet.
> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
> IVSHMEM: guest pci addr = e000, guest h/w addr = 2164588544, size =
> 2000
> pci_read_config: (val) 0xe004 <- 0x18 (addr)
> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
> IVSHMEM: guest pci addr = , guest h/w addr =
> 2164588544, size = 2000
> pci_read_config: (val) 0x <- 0x1c (addr)
> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x0 <- 0x20 (addr)
>
> the pci writes are all still 0, I can't see how my debug statements
> are incorrect though. Below is my trivial pci config debugging patch.
The debug out put should be before the for-loop.
for (i = 0; i < l && addr + i < config_size; val >>= 8, ++i) {
^
Here val becomes 0
uint8_t wmask = d->wmask[addr + i];
d->config[addr + i] = (d->config[addr + i] & ~wmask) | (val & wmask);
}
--
yamahata
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Aug 24, 2010 at 8:21 PM, Isaku Yamahata wrote:
> On Tue, Aug 24, 2010 at 10:52:36AM -0600, Cam Macdonell wrote:
>> Hi, 64-bit BARs still do not seem to be working.
>>
>> When using the latest seabios the guest does not hit a "BUG:"
>> statement, but booting still fails
>>
>> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
>> divide error: [#1] SMP
>> last sysfs file:
>> CPU 0
>> Modules linked in:
>>
>> Pid: 1, comm: swapper Not tainted 2.6.35+ #299 /Bochs
>> RIP: 0010:[] [] hpet_alloc+0x12c/0x35b
>> RSP: 0018:88007d7b3d80 EFLAGS: 00010246
>> RAX: 00038d7ea4c68000 RBX: 88007d062cc0 RCX:
>> RDX: RSI: RDI: 817bb9b0
>> RBP: 88007d7b3dc0 R08: 80d0 R09: c900
>> R10: 88007d72b5a0 R11: R12: 88007d7b3dd0
>> R13: c900 R14: R15: 817a41c3
>> FS: () GS:88000200() knlGS:
>> CS: 0010 DS: ES: CR0: 8005003b
>> CR2: CR3: 01a42000 CR4: 06f0
>> DR0: DR1: DR2:
>> DR3: DR6: 0ff0 DR7: 0400
>> Process swapper (pid: 1, threadinfo 88007d7b2000, task 88007d7b8000)
>> Stack:
>> 88007f43ab90 88007f43ab90 81ca6174 81b1f5e1
>> <0> 0100 0100
>> <0> 88007d7b3e80 810294ea fed0 c900
>> Call Trace:
>> [] ? hpet_late_init+0x0/0xea
>> [] hpet_reserve_platform_timers+0x10b/0x115
>> [] ? hpet_late_init+0x0/0xea
>> [] hpet_late_init+0x6b/0xea
>> [] ? hpet_late_init+0x0/0xea
>> [] do_one_initcall+0x5e/0x159
>> [] kernel_init+0x19a/0x228
>> [] kernel_thread_helper+0x4/0x10
>> [] ? kernel_init+0x0/0x228
>> [] ? kernel_thread_helper+0x0/0x10
>> Code: 89 1d ca b2 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 c3 41 7a 81 48
>> 8d 04 02 4c 89 f2 48 c7 c7 b0 b9 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
>> f7 f1 83 7b 30 01 48 c7 c1 86 1c 7d 81 49 0f 46 cf 48 89 43
>> RIP [] hpet_alloc+0x12c/0x35b
>> RSP
>> ---[ end trace a7919e7f17c0a725 ]---
>> Kernel panic - not syncing: Attempted to kill init!
>> Pid: 1, comm: swapper Tainted: G D 2.6.35+ #299
>> Call Trace:
>> [] panic+0x8b/0x10b
>> [] ? exit_ptrace+0x38/0x121
>> [] do_exit+0x7a/0x722
>> [] ? spin_unlock_irqrestore+0xe/0x10
>> [] ? kmsg_dump+0x12b/0x145
>> [] oops_end+0xbf/0xc7
>> [] die+0x5a/0x63
>> [] do_trap+0x121/0x130
>> [] do_divide_error+0x96/0x9f
>> [] ? hpet_alloc+0x12c/0x35b
>> [] ? radix_tree_preload+0x34/0x88
>> [] divide_error+0x1b/0x20
>> [] ? hpet_alloc+0x12c/0x35b
>> [] ? hpet_late_init+0x0/0xea
>> [] hpet_reserve_platform_timers+0x10b/0x115
>> [] ? hpet_late_init+0x0/0xea
>> [] hpet_late_init+0x6b/0xea
>> [] ? hpet_late_init+0x0/0xea
>> [] do_one_initcall+0x5e/0x159
>> [] kernel_init+0x19a/0x228
>> [] kernel_thread_helper+0x4/0x10
>> [] ? kernel_init+0x0/0x228
>> [] ? kernel_thread_helper+0x0/0x10
>>
>> seabios output for the device:
>>
>> PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
>> region 0: 0xf102
>> region 2: 0x
>> init smm
>>
>> Running the latest seabios, the debug output only remaps the BAR
>> twice, once with a potentially correct address of e
>>
>> pci_read_config: (val) 0xe004 <- 0x18 (addr)
>
> The upstream seabios lacks overflow check at the moment.
> I haven't found time to address PMM yet.
>
>
>> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
>> IVSHMEM: guest pci addr = e000, guest h/w addr = 2164588544, size =
>> 2000
>> pci_read_config: (val) 0xe004 <- 0x18 (addr)
>> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
>> IVSHMEM: guest pci addr = , guest h/w addr =
>> 2164588544, size = 2000
>> pci_read_config: (val) 0x <- 0x1c (addr)
>> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x0 <- 0x20 (addr)
>>
>> the pci writes are all still 0, I can't see how my debug statements
>> are incorrect though. Below is my trivial pci config debugging patch.
>
> The debug out put should be before the for-loop.
>
> for (i = 0; i < l && addr + i < config_size; val >>= 8, ++i) {
> ^
> Here val becomes 0
> uint8_t wmask = d->wmask[addr + i];
> d->config[addr + i] = (d->config[addr + i] & ~wmask) | (val & wmask);
> }
>
> --
> yamahata
>
>
Ah, thanks. I moved the debug statement and now the writes are the
proper values.
In seabios-kvm, it seems the guest is writing the address of c040 to
0x1c which results to the 48-bit address c040.
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <-
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Fri, Aug 27, 2010 at 01:35:23PM -0600, Cam Macdonell wrote: > In upstream seabios.git, the c040 is not written, but the device > returns from 0x1c (only reads and writes to 0x18 and 0x1c are > shown below) > > pci_read_config: (val) 0x4 <- 0x18 (addr) > pci_write_config: (val) 0x -> 0x18 (addr) > pci_read_config: (val) 0xe004 <- 0x18 (addr) > pci_write_config: (val) 0x4 -> 0x18 (addr) > pci_read_config: (val) 0x4 <- 0x18 (addr) this read is useless. I sent out the patch. > pci_write_config: (val) 0x0 -> 0x18 (addr) > pci_write_config: (val) 0x0 -> 0x1c (addr) It looks like that overflow occurred. You need overflow patch. As the size is huge, seabios can't map it. Anyway even with overflow patch, the expected result is to leave the bar unmodified(initial value). > pci_read_config: (val) 0x4 <- 0x18 (addr) > pci_write_config: (val) 0x -> 0x18 (addr) > IVSHMEM: guest pci addr = e000, guest h/w addr = 2164588544, size = > 2000 > pci_read_config: (val) 0xe004 <- 0x18 (addr) > pci_write_config: (val) 0x4 -> 0x18 (addr) > pci_read_config: (val) 0x0 <- 0x1c (addr) > pci_write_config: (val) 0x -> 0x1c (addr) > IVSHMEM: guest pci addr = , guest h/w addr = > 2164588544, size = 2000 > pci_read_config: (val) 0x <- 0x1c (addr) > pci_write_config: (val) 0x0 -> 0x1c (addr) Is the above done by guest OS? The guest OS doesn't seem to know 64bit bar. -- yamahata
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jun 29, 2010 at 11:48:13AM -0600, Cam Macdonell wrote:
> On Tue, Jun 29, 2010 at 12:50 AM, Avi Kivity wrote:
> > On 06/28/2010 11:38 PM, Cam Macdonell wrote:
> >>
> >>>
> > Is this really the address the guest programmed, or is qemu
> > misinterpreting
> > it?
> >
> >
> >>>
> >>> Well, what's the answer?
> >>>
> >>
> >> You're going to have to give me a hint on how to determine that.
> >>
> >> lspci in the guest shows the following
> >>
> >> Memory at c200 (64-bit, non-prefetchable) [size=1024M]
> >>
> >> does that demonstrate a guest generated address?
> >>
> >
> > That's the result of a round trip: the guest programmed the address and then
> > read it back. ?It could have been screwed up in the first place, or perhaps
> > qemu screwed it up.
> >
> > Add a printf() to the config space handlers in qemu (likely in your own
> > code) on writes and reads, and show the relevant writes (and reads) for this
> > BAR.
> >
> > That's the theory of deductive debugging; however browsing the code shows
> > the guest is at fault:
> >
> > ? ? ? ?for (i = 0; i < PCI_NUM_REGIONS; i++) {
> > ? ? ? ? ? ?int ofs;
> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT)
> > ? ? ? ? ? ? ? ?ofs = PCI_ROM_ADDRESS;
> > ? ? ? ? ? ?else
> > ? ? ? ? ? ? ? ?ofs = PCI_BASE_ADDRESS_0 + i * 4;
> >
> > ? ? ? ? ? ?u32 old = pci_config_readl(bdf, ofs);
> > ? ? ? ? ? ?u32 mask;
> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT) {
> > ? ? ? ? ? ? ? ?mask = PCI_ROM_ADDRESS_MASK;
> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, mask);
> > ? ? ? ? ? ?} else {
> > ? ? ? ? ? ? ? ?if (old & PCI_BASE_ADDRESS_SPACE_IO)
> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_IO_MASK;
> > ? ? ? ? ? ? ? ?else
> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_MEM_MASK;
> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, ~0);
> > ? ? ? ? ? ?}
> > ? ? ? ? ? ?u32 val = pci_config_readl(bdf, ofs);
> > ? ? ? ? ? ?pci_config_writel(bdf, ofs, old);
> >
> > ? ? ? ? ? ?if (val != 0) {
> > ? ? ? ? ? ? ? ?u32 size = (~(val & mask)) + 1;
> > ? ? ? ? ? ? ? ?if (val & PCI_BASE_ADDRESS_SPACE_IO)
> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_io_addr;
> > ? ? ? ? ? ? ? ?else
> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_mem_addr;
> > ? ? ? ? ? ? ? ?*paddr = ALIGN(*paddr, size);
> > ? ? ? ? ? ? ? ?pci_set_io_region_addr(bdf, i, *paddr);
> > ? ? ? ? ? ? ? ?*paddr += size;
> > ? ? ? ? ? ?}
> > ? ? ? ?}
> > ? ? ? ?break;
> > ? ?}
> >
> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
> > thinks the second half of the BAR is an I/O region instead of memory (hence
> > the c200, that's part of the pci portio region.
I've sent the patches to address it. But they haven't been merged yet.
seabios doesn't map BARs beyond 4GB.
If bar is mapped beyond 4GB, guest BIOS does it.
To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
in config.h of seabios
> > Do post those reads and writes, I think there's more than one thing wrong
> > here.
>
> Here it is, I added the debug statements to pci_read_config and
> pci_default_write_config.
>
> here are the reads and writes to offsets 0x18 and 0x1c where a 64-bit
> BAR2 config would be configured
It seems that 0x1c is accessed as BAR3.
The write value in the log is always 0.
Some comment in the log.
>
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0xc004 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0xc004 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0xc040 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
>
> the complete read/write profile is below along with debug statements
> from the map functions for the BARs (prefixed with IVSHMEM)
>
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x1110 <- 0x2 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x500 <- 0xa (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x1110 <- 0x2 (addr)
> pci_read_config: (val) 0x0 <- 0x10 (addr)
> pci_write_config: (val) 0x0 -> 0x10 (addr)
> pci_read_config: (val) 0xff00 <- 0x10 (addr)
> pci_write_config: (val) 0x0 -> 0x10 (addr)
> pci_read_config: (val) 0x0 <- 0x10 (addr)
> pci
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jun 29, 2010 at 9:29 PM, Isaku Yamahata wrote:
> On Tue, Jun 29, 2010 at 11:48:13AM -0600, Cam Macdonell wrote:
>> On Tue, Jun 29, 2010 at 12:50 AM, Avi Kivity wrote:
>> > On 06/28/2010 11:38 PM, Cam Macdonell wrote:
>> >>
>> >>>
>> > Is this really the address the guest programmed, or is qemu
>> > misinterpreting
>> > it?
>> >
>> >
>> >>>
>> >>> Well, what's the answer?
>> >>>
>> >>
>> >> You're going to have to give me a hint on how to determine that.
>> >>
>> >> lspci in the guest shows the following
>> >>
>> >> Memory at c200 (64-bit, non-prefetchable) [size=1024M]
>> >>
>> >> does that demonstrate a guest generated address?
>> >>
>> >
>> > That's the result of a round trip: the guest programmed the address and
>> > then
>> > read it back. ?It could have been screwed up in the first place, or perhaps
>> > qemu screwed it up.
>> >
>> > Add a printf() to the config space handlers in qemu (likely in your own
>> > code) on writes and reads, and show the relevant writes (and reads) for
>> > this
>> > BAR.
>> >
>> > That's the theory of deductive debugging; however browsing the code shows
>> > the guest is at fault:
>> >
>> > ? ? ? ?for (i = 0; i < PCI_NUM_REGIONS; i++) {
>> > ? ? ? ? ? ?int ofs;
>> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT)
>> > ? ? ? ? ? ? ? ?ofs = PCI_ROM_ADDRESS;
>> > ? ? ? ? ? ?else
>> > ? ? ? ? ? ? ? ?ofs = PCI_BASE_ADDRESS_0 + i * 4;
>> >
>> > ? ? ? ? ? ?u32 old = pci_config_readl(bdf, ofs);
>> > ? ? ? ? ? ?u32 mask;
>> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT) {
>> > ? ? ? ? ? ? ? ?mask = PCI_ROM_ADDRESS_MASK;
>> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, mask);
>> > ? ? ? ? ? ?} else {
>> > ? ? ? ? ? ? ? ?if (old & PCI_BASE_ADDRESS_SPACE_IO)
>> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_IO_MASK;
>> > ? ? ? ? ? ? ? ?else
>> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_MEM_MASK;
>> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, ~0);
>> > ? ? ? ? ? ?}
>> > ? ? ? ? ? ?u32 val = pci_config_readl(bdf, ofs);
>> > ? ? ? ? ? ?pci_config_writel(bdf, ofs, old);
>> >
>> > ? ? ? ? ? ?if (val != 0) {
>> > ? ? ? ? ? ? ? ?u32 size = (~(val & mask)) + 1;
>> > ? ? ? ? ? ? ? ?if (val & PCI_BASE_ADDRESS_SPACE_IO)
>> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_io_addr;
>> > ? ? ? ? ? ? ? ?else
>> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_mem_addr;
>> > ? ? ? ? ? ? ? ?*paddr = ALIGN(*paddr, size);
>> > ? ? ? ? ? ? ? ?pci_set_io_region_addr(bdf, i, *paddr);
>> > ? ? ? ? ? ? ? ?*paddr += size;
>> > ? ? ? ? ? ?}
>> > ? ? ? ?}
>> > ? ? ? ?break;
>> > ? ?}
>> >
>> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
>> > thinks the second half of the BAR is an I/O region instead of memory (hence
>> > the c200, that's part of the pci portio region.
>
> I've sent the patches to address it. But they haven't been merged yet.
> seabios doesn't map BARs beyond 4GB.
> If bar is mapped beyond 4GB, guest BIOS does it.
Have those patches been merged yet?
>
>
> To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> in config.h of seabios
Where does the output from seabios end up? Inside dmesg?
>
>
>> > Do post those reads and writes, I think there's more than one thing wrong
>> > here.
>>
>> Here it is, I added the debug statements to pci_read_config and
>> pci_default_write_config.
>>
>> here are the reads and writes to offsets 0x18 and 0x1c where a 64-bit
>> BAR2 config would be configured
>
> It seems that 0x1c is accessed as BAR3.
> The write value in the log is always 0.
>
> Some comment in the log.
>
>>
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0xc004 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0xc004 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0xc040 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>>
>> the complete read/write profile is below along with debug statements
>> from the map functions for the BARs (prefixed with IVSHMEM)
>>
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x1110 <- 0x2 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x500 <- 0xa (addr)
>> pci_read_config: (val)
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote: > >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also > >> > thinks the second half of the BAR is an I/O region instead of memory > >> > (hence > >> > the c200, that's part of the pci portio region. > > > > I've sent the patches to address it. But they haven't been merged yet. > > seabios doesn't map BARs beyond 4GB. > > If bar is mapped beyond 4GB, guest BIOS does it. > > Have those patches been merged yet? They have been merged into seabios upstream now. qemu seabios fork hasn't pulled for a while, though. > > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL > > in config.h of seabios > > Where does the output from seabios end up? Inside dmesg? It outputs them to the serial console which qemu emulates. seabios is out of kernel control, so dmesg doesn't show it. > >> pci_read_config: (val) 0x0 <- 0x1c (addr) > >> pci_write_config: (val) 0x0 -> 0x1c (addr) > >> pci_read_config: (val) 0x <- 0x1c (addr) > >> pci_write_config: (val) 0x0 -> 0x1c (addr) > >> pci_read_config: (val) 0x0 <- 0x1c (addr) > >> pci_write_config: (val) 0x0 -> 0x1c (addr) > > > > seabios BAR3. Not sure how it is mapped from this > > message. > > Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and > BAR3 to store all 64-bits? Yes. Seabios misbehaves. 64bit bar is(was) a missing feature. -- yamahata
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata wrote: > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote: >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also >> >> > thinks the second half of the BAR is an I/O region instead of memory >> >> > (hence >> >> > the c200, that's part of the pci portio region. >> > >> > I've sent the patches to address it. But they haven't been merged yet. >> > seabios doesn't map BARs beyond 4GB. >> > If bar is mapped beyond 4GB, guest BIOS does it. >> >> Have those patches been merged yet? > > They have been merged into seabios upstream now. > qemu seabios fork hasn't pulled for a while, though. > > >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL >> > in config.h of seabios >> >> Where does the output from seabios end up? Inside dmesg? > > It outputs them to the serial console which qemu emulates. > seabios is out of kernel control, so dmesg doesn't show it. > > >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) >> >> pci_read_config: (val) 0x <- 0x1c (addr) >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) >> > >> > seabios BAR3. Not sure how it is mapped from this >> > message. >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and >> BAR3 to store all 64-bits? > > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature. > -- > yamahata > > With the latest seabios git passed via -bios, I no longer see the 48-bit address, but instead a 32-bit address and then . This guest has 1gb of RAM so the address isn't be mapped beyond 4g. IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size = 2000 IVSHMEM: guest pci addr = , guest h/w addr = 1090912256, size = 2000 However, booting fails when I use a 64-bit BAR. Booting is fine with a 32-bit BAR. HPET: 1 timers in total, 0 timers will be used for per-cpu timer divide error: [#1] SMP last sysfs file: CPU 0 Modules linked in: Pid: 1, comm: swapper Not tainted 2.6.35-rc1 #280 /Bochs RIP: 0010:[] [] hpet_alloc+0x12c/0x35b RSP: 0018:88003e61fd80 EFLAGS: 00010246 RAX: 00038d7ea4c68000 RBX: 88003e6efd80 RCX: RDX: RSI: RDI: 817b8520 RBP: 88003e61fdc0 R08: 80d0 R09: c900 R10: 88003f9ac600 R11: R12: 88003e61fdd0 R13: c900 R14: R15: 817a00a9 FS: () GS:88000200() knlGS: CS: 0010 DS: ES: CR0: 8005003b CR2: CR3: 01a42000 CR4: 06f0DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Process swapper (pid: 1, threadinfo 88003e61e000, task 88003e62) Stack: 88003f43ab90 88003f43ab90 81ca3174 81b1d596 <0> 0100 0100 <0> 88003e61fe80 8102945d fed0 c900 Call Trace: [] ? hpet_late_init+0x0/0xea [] hpet_reserve_platform_timers+0x10b/0x115 [] ? hpet_late_init+0x0/0xea [] hpet_late_init+0x6b/0xea [] ? hpet_late_init+0x0/0xea [] do_one_initcall+0x5e/0x159 [] kernel_init+0x18e/0x21c [] kernel_thread_helper+0x4/0x10 [] ? kernel_init+0x0/0x21c [] ? kernel_thread_helper+0x0/0x10 Code: 89 1d 8a 92 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 a9 00 7a 81 48 8d 04 02 4c 89 f2 48 c7 c7 20 85 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48> f7 f1 83 7b 30 01 48 c7 c1 cd fa 7c 81 49 0f 46 cf 48 89 43 RIP [] hpet_alloc+0x12c/0x35b RSP ---[ end trace a7919e7f17c0a725 ]--- Kernel panic - not syncing: Attempted to kill init! Pid: 1, comm: swapper Tainted: G D 2.6.35-rc1 #280 Call Trace: [] panic+0x8b/0x10b [] ? exit_ptrace+0x38/0x121 [] do_exit+0x7a/0x722 [] ? spin_unlock_irqrestore+0xe/0x10 [] ? kmsg_dump+0x12b/0x145 [] oops_end+0xbf/0xc7 [] die+0x5a/0x63 [] do_trap+0x121/0x130 [] do_divide_error+0x96/0x9f [] ? hpet_alloc+0x12c/0x35b [] divide_error+0x1b/0x20 [] ? hpet_alloc+0x12c/0x35b [] ? hpet_late_init+0x0/0xea [] hpet_reserve_platform_timers+0x10b/0x115 [] ? hpet_late_init+0x0/0xea [] hpet_late_init+0x6b/0xea [] ? hpet_late_init+0x0/0xea [] do_one_initcall+0x5e/0x159 [] kernel_init+0x18e/0x21c [] kernel_thread_helper+0x4/0x10 [] ? kernel_init+0x0/0x21c [] ? kernel_thread_helper+0x0/0x10
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata
> wrote:
> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it
> >> >> > also
> >> >> > thinks the second half of the BAR is an I/O region instead of memory
> >> >> > (hence
> >> >> > the c200, that's part of the pci portio region.
> >> >
> >> > I've sent the patches to address it. But they haven't been merged yet.
> >> > seabios doesn't map BARs beyond 4GB.
> >> > If bar is mapped beyond 4GB, guest BIOS does it.
> >>
> >> Have those patches been merged yet?
> >
> > They have been merged into seabios upstream now.
> > qemu seabios fork hasn't pulled for a while, though.
> >
> >
> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> >> > in config.h of seabios
> >>
> >> Where does the output from seabios end up? ?Inside dmesg?
> >
> > It outputs them to the serial console which qemu emulates.
> > seabios is out of kernel control, so dmesg doesn't show it.
> >
> >
> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> pci_read_config: (val) 0x <- 0x1c (addr)
> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >
> >> > seabios BAR3. Not sure how it is mapped from this
> >> > message.
> >>
> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
> >> BAR3 to store all 64-bits?
> >
> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
> > --
> > yamahata
> >
> >
>
> With the latest seabios git passed via -bios, I no longer see the
> 48-bit address, but instead a 32-bit address and then
> . This guest has 1gb of RAM so the address isn't be
> mapped beyond 4g.
Can I see the debug log like before?
(hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)
Do you know who sets the BAR to ?
If it's seabios, does the following patch help?
diff --git a/src/pciinit.c b/src/pciinit.c
index b110531..ce07709 100644
--- a/src/pciinit.c
+++ b/src/pciinit.c
@@ -117,11 +117,7 @@ static int pci_bios_allocate_region(u16 bdf, int
region_num)
int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
(val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == PCI_BASE_ADDRESS_MEM_TYPE_64;
if (is_64bit) {
-if (size > 0) {
-pci_config_writel(bdf, ofs + 4, 0);
-} else {
-pci_config_writel(bdf, ofs + 4, ~0);
-}
+pci_config_writel(bdf, ofs + 4, 0);
}
return is_64bit;
}
> IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size =
> 2000
> IVSHMEM: guest pci addr = , guest h/w addr =
> 1090912256, size = 2000
>
> However, booting fails when I use a 64-bit BAR. Booting is fine with
> a 32-bit BAR.
>
> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
> divide error: [#1] SMP last sysfs file:
> CPU 0
> Modules linked in:
>
> Pid: 1, comm: swapper Not tainted 2.6.35-rc1 #280 /Bochs
> RIP: 0010:[] [] hpet_alloc+0x12c/0x35b
> RSP: 0018:88003e61fd80 EFLAGS: 00010246
> RAX: 00038d7ea4c68000 RBX: 88003e6efd80 RCX:
> RDX: RSI: RDI: 817b8520
> RBP: 88003e61fdc0 R08: 80d0 R09: c900
> R10: 88003f9ac600 R11: R12: 88003e61fdd0
> R13: c900 R14: R15: 817a00a9
> FS: () GS:88000200() knlGS:
> CS: 0010 DS: ES: CR0: 8005003b
> CR2: CR3: 01a42000 CR4: 06f0DR0:
> DR1: DR2:
> DR3: DR6: 0ff0 DR7: 0400
> Process swapper (pid: 1, threadinfo 88003e61e000, task 88003e62)
> Stack:
> 88003f43ab90 88003f43ab90 81ca3174 81b1d596
> <0> 0100 0100
> <0> 88003e61fe80 8102945d fed0 c900
> Call Trace:
> [] ? hpet_late_init+0x0/0xea
> [] hpet_reserve_platform_timers+0x10b/0x115
> [] ? hpet_late_init+0x0/0xea
> [] hpet_late_init+0x6b/0xea
> [] ? hpet_late_init+0x0/0xea
> [] do_one_initcall+0x5e/0x159
> [] kernel_init+0x18e/0x21c
> [] kernel_thread_helper+0x4/0x10
> [] ? kernel_init+0x0/0x21c
> [] ? kernel_thread_helper+0x0/0x10
> Code: 89 1d 8a 92 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 a9 00 7a 81 48
> 8d 04 02 4c 89 f2 48 c7 c7 20 85 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
> f7 f1 83 7b 30 01 48
> c7 c1 cd fa 7c 81 49 0f 46 cf 48 89 43
> RIP [] hpet_alloc+0x12c/0x35b
> RSP
> ---[ end trace a7919e7f17c0a725 ]---
> Kernel panic - not syncing: Attempted to kill init!
> Pid: 1, comm: swapper Tainted: G D 2
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata wrote:
> On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
>> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata
>> wrote:
>> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
>> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it
>> >> >> > also
>> >> >> > thinks the second half of the BAR is an I/O region instead of memory
>> >> >> > (hence
>> >> >> > the c200, that's part of the pci portio region.
>> >> >
>> >> > I've sent the patches to address it. But they haven't been merged yet.
>> >> > seabios doesn't map BARs beyond 4GB.
>> >> > If bar is mapped beyond 4GB, guest BIOS does it.
>> >>
>> >> Have those patches been merged yet?
>> >
>> > They have been merged into seabios upstream now.
>> > qemu seabios fork hasn't pulled for a while, though.
>> >
>> >
>> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
>> >> > in config.h of seabios
>> >>
>> >> Where does the output from seabios end up? ?Inside dmesg?
>> >
>> > It outputs them to the serial console which qemu emulates.
>> > seabios is out of kernel control, so dmesg doesn't show it.
>> >
>> >
>> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> >> pci_read_config: (val) 0x <- 0x1c (addr)
>> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> >
>> >> > seabios BAR3. Not sure how it is mapped from this
>> >> > message.
>> >>
>> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
>> >> BAR3 to store all 64-bits?
>> >
>> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
>> > --
>> > yamahata
>> >
>> >
>>
>> With the latest seabios git passed via -bios, I no longer see the
>> 48-bit address, but instead a 32-bit address and then
>> . This guest has 1gb of RAM so the address isn't be
>> mapped beyond 4g.
>
> Can I see the debug log like before?
> (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)
Here's the dump from SeaBIOS in the region related to the PCI devices.
The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit.
PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8
region 0: 0xf000
region 1: 0xf200
region 6: 0xf201
PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000
region 0: 0xc020
region 1: 0xf202
region 6: 0xf203
PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
region 0: 0xf204
region 1: 0xf2041000
region 2: 0x
>
> Do you know who sets the BAR to ?
Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM'
lines are from the map function passed to pci_register_bar(). It
looks like SeaBIOS sets the address to 0 and then the potentially
useful e000 address gets mangled into 00.
IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912
...snip...
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size = 2000
pci_read_config: (val) 0xe004 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
IVSHMEM: guest pci addr = , guest h/w addr =
1090912256, size = 2000
pci_read_config: (val) 0x <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
and with the 64-bit guest I get this error as well (recall the guest
fails to boot on 64-bit)
BUG: kvm_dirty_pages_log_change: invalid parameters
f000-f0ff
> If it's seabios, does the following patch help?
The patch doesn't make a difference. Perhaps it's Qemu then?
>
> diff --git a/src/pciinit.c b/src/pciinit.c
> index b110531..ce07709 100644
> --- a/src/pciinit.c
> +++ b/src/pciinit.c
> @@ -117,11 +117,7 @@ static int pci_bios_allocate_region(u16 bdf, int
> region_num)
> int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
> (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
> PCI_BASE_ADDRESS_MEM_TYPE_64;
> if (is_64bit) {
> - if (size > 0) {
> - pci_config_writel(bdf, ofs + 4, 0);
> - } else {
> - pci_config_writel(bdf, ofs + 4, ~0);
> - }
> + pci_config_writel(bdf, ofs + 4, 0);
> }
> return is_64bit;
> }
>
>
>
>> IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size =
>> 2000
>> IVSHMEM: guest pci addr = , guest h/w addr =
>> 1090912256, size = 2000
>>
>> However, booting fails when I use a 64-bit BAR. Booting is fine with
>> a 32-bit BAR.
>>
>> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
>> divide error: [#1] SMP last sysfs file:
>> CPU 0
>> Modules linked in:
>>
>> Pid: 1, comm: swapper Not tainted 2.6.35-rc1 #280 /Bochs
>> RIP: 0010:[] [] h
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote:
> On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata
> wrote:
> > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
> >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata
> >> wrote:
> >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
> >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like
> >> >> >> > it also
> >> >> >> > thinks the second half of the BAR is an I/O region instead of
> >> >> >> > memory (hence
> >> >> >> > the c200, that's part of the pci portio region.
> >> >> >
> >> >> > I've sent the patches to address it. But they haven't been merged yet.
> >> >> > seabios doesn't map BARs beyond 4GB.
> >> >> > If bar is mapped beyond 4GB, guest BIOS does it.
> >> >>
> >> >> Have those patches been merged yet?
> >> >
> >> > They have been merged into seabios upstream now.
> >> > qemu seabios fork hasn't pulled for a while, though.
> >> >
> >> >
> >> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> >> >> > in config.h of seabios
> >> >>
> >> >> Where does the output from seabios end up? ?Inside dmesg?
> >> >
> >> > It outputs them to the serial console which qemu emulates.
> >> > seabios is out of kernel control, so dmesg doesn't show it.
> >> >
> >> >
> >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> >> pci_read_config: (val) 0x <- 0x1c (addr)
> >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> >
> >> >> > seabios BAR3. Not sure how it is mapped from this
> >> >> > message.
> >> >>
> >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
> >> >> BAR3 to store all 64-bits?
> >> >
> >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
> >> > --
> >> > yamahata
> >> >
> >> >
> >>
> >> With the latest seabios git passed via -bios, I no longer see the
> >> 48-bit address, but instead a 32-bit address and then
> >> . ?This guest has 1gb of RAM so the address isn't be
> >> mapped beyond 4g.
> >
> > Can I see the debug log like before?
> > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)
>
> Here's the dump from SeaBIOS in the region related to the PCI devices.
> The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit.
>
> PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8
> region 0: 0xf000
> region 1: 0xf200
> region 6: 0xf201
> PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000
> region 0: 0xc020
> region 1: 0xf202
> region 6: 0xf203
> PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
> region 0: 0xf204
> region 1: 0xf2041000
> region 2: 0x
Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110)
the BAR in quistion?
The value 0 seems odd. Probably BAR address calculation overflowed.
Currently seabios doesn't check overflow. I attached the patch.
> > Do you know who sets the BAR to ?
>
> Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM'
> lines are from the map function passed to pci_register_bar(). It
> looks like SeaBIOS sets the address to 0 and then the potentially
> useful e000 address gets mangled into 00.
There is something wrong with the debug message of write case, I suppose.
All written value are 0, but the resulted effect doesn't seems so.
>
> IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912
>
> ...snip...
>
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size =
> 2000
If 0 is written to 0x18, the bar address should be 0, but it says e000.
> pci_read_config: (val) 0xe004 <- 0x18 (addr)
The read value isn't 0. and so on...
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> IVSHMEM: guest pci addr = , guest h/w addr =
> 1090912256, size = 2000
> pci_read_config: (val) 0x <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
>
> and with the 64-bit guest I get this error as well (recall the guest
> fails to boot on 64-bit)
>
> BUG: kvm_dirty_pages_log_change: invalid parameters
> f000-f0ff
diff --git a/src/pciinit.c b/src/pciinit.c
index b110531..6eca2ce 100644
--- a/src/pciinit.c
+++ b/src/pciinit.c
@@ -90,7 +90,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
/* If pci_bios_prefmem_addr == 0, keep old behaviour */
pci_bios_prefmem_addr != 0) {
paddr = &pci_bios_prefmem_addr;
-if (ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
+if (ALIGN(*paddr, size) + size < *paddr ||
+
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
On Tue, Jul 20, 2010 at 06:52:23PM +0900, Isaku Yamahata wrote: > On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote: > > On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata > > wrote: > > > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote: > > >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata > > >> wrote: > > >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote: > > >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like > > >> >> >> > it also > > >> >> >> > thinks the second half of the BAR is an I/O region instead of > > >> >> >> > memory (hence > > >> >> >> > the c200, that's part of the pci portio region. > > >> >> > > > >> >> > I've sent the patches to address it. But they haven't been merged > > >> >> > yet. > > >> >> > seabios doesn't map BARs beyond 4GB. > > >> >> > If bar is mapped beyond 4GB, guest BIOS does it. > > >> >> > > >> >> Have those patches been merged yet? > > >> > > > >> > They have been merged into seabios upstream now. > > >> > qemu seabios fork hasn't pulled for a while, though. > > >> > > > >> > > > >> >> > To see how seabios works, it would help to increase > > >> >> > CONFIG_DEBUG_LEVEL > > >> >> > in config.h of seabios > > >> >> > > >> >> Where does the output from seabios end up? ?Inside dmesg? > > >> > > > >> > It outputs them to the serial console which qemu emulates. > > >> > seabios is out of kernel control, so dmesg doesn't show it. > > >> > > > >> > > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) > > >> >> >> pci_read_config: (val) 0x <- 0x1c (addr) > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) > > >> >> > > > >> >> > seabios BAR3. Not sure how it is mapped from this > > >> >> > message. > > >> >> > > >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and > > >> >> BAR3 to store all 64-bits? > > >> > > > >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature. > > >> > -- > > >> > yamahata > > >> > > > >> > > > >> > > >> With the latest seabios git passed via -bios, I no longer see the > > >> 48-bit address, but instead a 32-bit address and then > > >> . ?This guest has 1gb of RAM so the address isn't be > > >> mapped beyond 4g. > > > > > > Can I see the debug log like before? > > > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.) > > > > Here's the dump from SeaBIOS in the region related to the PCI devices. > > The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit. > > > > PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8 > > region 0: 0xf000 > > region 1: 0xf200 > > region 6: 0xf201 > > PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000 > > region 0: 0xc020 > > region 1: 0xf202 > > region 6: 0xf203 > > PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110 > > region 0: 0xf204 > > region 1: 0xf2041000 > > region 2: 0x > > Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110) > the BAR in quistion? > The value 0 seems odd. Probably BAR address calculation overflowed. > Currently seabios doesn't check overflow. I attached the patch. > > > > > Do you know who sets the BAR to ? > > > > Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM' > > lines are from the map function passed to pci_register_bar(). It > > looks like SeaBIOS sets the address to 0 and then the potentially > > useful e000 address gets mangled into 00. > > There is something wrong with the debug message of write case, I suppose. > All written value are 0, but the resulted effect doesn't seems so. > > > > > IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912 > > > > ...snip... > > > > pci_read_config: (val) 0x4 <- 0x18 (addr) > > pci_write_config: (val) 0x0 -> 0x18 (addr) > > IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size = > > 2000 > > If 0 is written to 0x18, the bar address should be 0, but it says e000. > > > pci_read_config: (val) 0xe004 <- 0x18 (addr) > > The read value isn't 0. and so on... > > > pci_write_config: (val) 0x0 -> 0x18 (addr) > > pci_read_config: (val) 0x0 <- 0x1c (addr) > > pci_write_config: (val) 0x0 -> 0x1c (addr) > > IVSHMEM: guest pci addr = , guest h/w addr = > > 1090912256, size = 2000 > > pci_read_config: (val) 0x <- 0x1c (addr) > > pci_write_config: (val) 0x0 -> 0x1c (addr) > > > > and with the 64-bit guest I get this error as well (recall the guest > > fails to boot on 64-bit) > > > > BUG: kvm_dirty_pages_log_change: invalid parameters > > f000-f0ff > > > diff --git a/src/pciinit.c b/src/pciinit.c > index b110531..6eca2ce 100644 > --- a/src/pciinit.c > +++ b/src/pciinit.c > @@ -90,7 +90,8 @@ static int pci_bios
Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
Added Cc: seab...@seabios.org On Wed, Jul 21, 2010 at 06:31:01AM +0300, Michael S. Tsirkin wrote: > On Tue, Jul 20, 2010 at 06:52:23PM +0900, Isaku Yamahata wrote: > > On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote: > > > On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata > > > wrote: > > > > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote: > > > >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata > > > >> wrote: > > > >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote: > > > >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks > > > >> >> >> > like it also > > > >> >> >> > thinks the second half of the BAR is an I/O region instead of > > > >> >> >> > memory (hence > > > >> >> >> > the c200, that's part of the pci portio region. > > > >> >> > > > > >> >> > I've sent the patches to address it. But they haven't been merged > > > >> >> > yet. > > > >> >> > seabios doesn't map BARs beyond 4GB. > > > >> >> > If bar is mapped beyond 4GB, guest BIOS does it. > > > >> >> > > > >> >> Have those patches been merged yet? > > > >> > > > > >> > They have been merged into seabios upstream now. > > > >> > qemu seabios fork hasn't pulled for a while, though. > > > >> > > > > >> > > > > >> >> > To see how seabios works, it would help to increase > > > >> >> > CONFIG_DEBUG_LEVEL > > > >> >> > in config.h of seabios > > > >> >> > > > >> >> Where does the output from seabios end up? ?Inside dmesg? > > > >> > > > > >> > It outputs them to the serial console which qemu emulates. > > > >> > seabios is out of kernel control, so dmesg doesn't show it. > > > >> > > > > >> > > > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) > > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) > > > >> >> >> pci_read_config: (val) 0x <- 0x1c (addr) > > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) > > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr) > > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr) > > > >> >> > > > > >> >> > seabios BAR3. Not sure how it is mapped from this > > > >> >> > message. > > > >> >> > > > >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 > > > >> >> and > > > >> >> BAR3 to store all 64-bits? > > > >> > > > > >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature. > > > >> > -- > > > >> > yamahata > > > >> > > > > >> > > > > >> > > > >> With the latest seabios git passed via -bios, I no longer see the > > > >> 48-bit address, but instead a 32-bit address and then > > > >> . ?This guest has 1gb of RAM so the address isn't be > > > >> mapped beyond 4g. > > > > > > > > Can I see the debug log like before? > > > > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.) > > > > > > Here's the dump from SeaBIOS in the region related to the PCI devices. > > > The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit. > > > > > > PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8 > > > region 0: 0xf000 > > > region 1: 0xf200 > > > region 6: 0xf201 > > > PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000 > > > region 0: 0xc020 > > > region 1: 0xf202 > > > region 6: 0xf203 > > > PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110 > > > region 0: 0xf204 > > > region 1: 0xf2041000 > > > region 2: 0x > > > > Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110) > > the BAR in quistion? > > The value 0 seems odd. Probably BAR address calculation overflowed. > > Currently seabios doesn't check overflow. I attached the patch. > > > > > > > > Do you know who sets the BAR to ? > > > > > > Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM' > > > lines are from the map function passed to pci_register_bar(). It > > > looks like SeaBIOS sets the address to 0 and then the potentially > > > useful e000 address gets mangled into 00. > > > > There is something wrong with the debug message of write case, I suppose. > > All written value are 0, but the resulted effect doesn't seems so. > > > > > > > > IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912 > > > > > > ...snip... > > > > > > pci_read_config: (val) 0x4 <- 0x18 (addr) > > > pci_write_config: (val) 0x0 -> 0x18 (addr) > > > IVSHMEM: guest pci addr = e000, guest h/w addr = 1090912256, size = > > > 2000 > > > > If 0 is written to 0x18, the bar address should be 0, but it says e000. > > > > > pci_read_config: (val) 0xe004 <- 0x18 (addr) > > > > The read value isn't 0. and so on... > > > > > pci_write_config: (val) 0x0 -> 0x18 (addr) > > > pci_read_config: (val) 0x0 <- 0x1c (addr) > > > pci_write_config: (val) 0x0 -> 0x1c (addr) > > > IVSHMEM: guest pci addr = , guest h/w addr = > > > 1090912256, size = 2000 > > > pci_read_config: (val) 0x <- 0x1c (addr) > > > pci_write_config: (val) 0x0 -> 0x1c (addr) > > > > > > and with the
