Thanks Andrew,
I did a simple experiment on Dellr720 server for allocating PCIe DMA
capable memory. This server has 64GB of physical memory.
First experiment:
--------------------------
In option ROM driver start function, i tried allocating different sizes
of memory allocation ( from 128MB to 4GB of memory) with PCI AllocateBuffer
and Map function. Driver successfully allocated and maped up to 2GB. But
It failed to allocate 3GB of memory.
Attached the driver source code file.
Here is my question:
1) Why i am not able to allocate more than 3GB of memory eventhough system
has 64GB of physical memory?
Thanks
Sateesh
On Thu, May 9, 2013 at 10:07 PM, Andrew Fish <[email protected]> wrote:
>
> On May 9, 2013, at 6:50 AM, satish kondapalli <[email protected]>
> wrote:
>
> > Hi,
> >
> > I have DellR720 server with 16GB of physical Memory. I am testing the
> option ROM driver on this server and my driver is executed successfully.
> But while booting Linux from hard disk it failed with the following error
> message on screen
> > "can not allocate memory Failed to allocate scratch mem!".
> >
> > If i remove my card from server its booted fine.
> >
> > Here is my question:
> > 1) In UEFI, is there any memory allocation size restriction?(In my
> driver i allocated EfiBootServicesData memory.)
> > 2) is it possible to allocate 4GB of pcie DMA capable memory? In my case
> it failed for 64MB.
>
> The only legal way to allocate a DMA buffer is to use
> EFI_PCI_IO_PROTOCOL.AllocateBuffer() and then call
> EFI_PCI_ROOT_BRIDGE_IO_PROTOCOL.Map() with an Operation like
> EfiPciOperationBusMasterCommonBuffer64.
>
>
> > 3) if i want a 4GB of PCIe DMA capable memory how to allocate?
> >
>
>
> 13.4 EFI PCI I/O Protocol - EFI_PCI_ROOT_BRIDGE_IO_PROTOCOL.Map()
>
> Bus master operations that require both read and write access or require
> multiple host device interactions within the same mapped region must use
> EfiPciOperation- BusMasterCommonBuffer or
> EfiPciOperationBusMasterCommonBuffer64. However, only memory allocated via
> the AllocateBuffer() interface can be mapped for this type of operation.
>
> In all mapping requests the resulting NumberOfBytes actually mapped may be
> less than the requested amount. In this case, the DMA operation will have
> to be broken up into smaller chunks. The Map() function will map as much of
> the DMA operation as it can at one time. The caller may have to loop on
> Map() and Unmap() in order to complete a large DMA transfer.
>
>
> Note: If you map as EfiPciOperationBusMasterCommonBuffer or
> EfiPciOperationBusMasterCommonBuffer64 you don't need to Map() and Unmap()
> on each DMA transfer.
>
> Thanks,
>
> Andrew Fish
>
> PS DMA coherency is for the most part handled in hardware on a PC so not
> following the rules may appear to work. But if you port that driver to ARM
> it will fail horribly. Not to mention a massively parallel high end X64
> system could fail too if you don't follow the rules.
>
> PPS Don't assume that the address of the DMA buffer is the same from the
> CPU and PCI perspective. The PCI card needs to use DeviceAddress, and it
> does not have to be the same as the HostAddress. The EFI DMA flows our
> outlined in13.2 PCI Root Bridge I/O Protocol:
>
> DMA Bus Master Read Operation
> • Call Map() for EfiPciOperationBusMasterRead or
> EfiPciOperationBusMasterRead64.
> • Program the DMA Bus Master with the DeviceAddress returned by Map().
> • Start the DMA Bus Master.
> • Wait for DMA Bus Master to complete the read operation.
> • Call Unmap().
>
> DMA Bus Master Write Operation
> • Call Map() for EfiPciOperationBusMasterWrite or
> EfiPciOperationBusMasterRead64.
> • Program the DMA Bus Master with the DeviceAddress returned by Map().
> • Start the DMA Bus Master.
> • Wait for DMA Bus Master to complete the write operation.
> • Perform a PCI controller specific read transaction to flush all PCI
> write buffers (See PCI Specification Section 3.2.5.2) .
> • Call Flush().
> • Call Unmap().
>
> DMA Bus Master Common Buffer Operation
> • Call AllocateBuffer() to allocate a common buffer.
> • Call Map() for EfiPciOperationBusMasterCommonBuffer or
> EfiPciOperationBusMasterCommonBuffer64.
> • Program the DMA Bus Master with the DeviceAddress returned by Map().
> • The common buffer can now be accessed equally by the processor and the
> DMA bus master.
> • Call Unmap().
> • Call FreeBuffer().
>
>
> > Thanks
> > Sateesh
> >
> >
> ------------------------------------------------------------------------------
> > Learn Graph Databases - Download FREE O'Reilly Book
> > "Graph Databases" is the definitive new guide to graph databases and
> > their applications. This 200-page book is written by three acclaimed
> > leaders in the field. The early access version is available now.
> > Download your free book today!
> http://p.sf.net/sfu/neotech_d2d_may_______________________________________________
> > edk2-devel mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/edk2-devel
>
>
>
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and
> their applications. This 200-page book is written by three acclaimed
> leaders in the field. The early access version is available now.
> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
> _______________________________________________
> edk2-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/edk2-devel
>
/*
* Copyright (c) 2009-2011
* Virident Systems, Inc., Milpitas, CA 95034
*
* Copying is strictly prohibited without explicit written permission
* from Virident Systems, Inc.
*
* Description:
* This file is to validate the Vendor ID, Device ID and registers
* BLOCK_IO_PROTOCOL for MC-NOR flash.
*
*/
#include "SpDrv.h"
EFI_DRIVER_BINDING_PROTOCOL gSpDriverBinding = {
SpDriverBindingSupported,
SpDriverBindingStart,
SpDriverBindingStop,
0xa,
NULL,
NULL
};
/**
The user code starts with this function.
@param ImageHandle The firmware allocated handle for the EFI image.
@param SystemTable A pointer to the EFI System Table.
@retval EFI_SUCCESS The entry point is executed successfully.
@retval other Some error occurs when executing this entry point.
**/
EFI_STATUS
EFIAPI
SpMain(
IN EFI_HANDLE ImageHandle,
IN EFI_SYSTEM_TABLE *SystemTable
)
{
EFI_STATUS Status=0;
//
// Install driver model protocol(s).
//
Status = EfiLibInstallDriverBinding (
ImageHandle,
SystemTable,
&gSpDriverBinding,
ImageHandle);
ASSERT_EFI_ERROR (Status);
return Status;
}
/**
Test to see if this driver supports ControllerHandle.
This service is called by the EFI boot service ConnectController(). In order
to make drivers as small as possible, there are a few calling restrictions for
this service. ConnectController() must follow these calling restrictions.
If any other agent wishes to call Supported() it must also follow these
calling restrictions.
@param This Protocol instance pointer.
@param ControllerHandle Handle of device to test
@param RemainingDevicePath Optional parameter use to pick a specific child
device to start.
@retval EFI_SUCCESS This driver supports this device
@retval EFI_ALREADY_STARTED This driver is already running on this device
@retval other This driver does not support this device
**/
EFI_STATUS
EFIAPI
SpDriverBindingSupported (
IN EFI_DRIVER_BINDING_PROTOCOL *This,
IN EFI_HANDLE Controller,
IN EFI_DEVICE_PATH_PROTOCOL *RemainingDevicePath OPTIONAL
)
{
EFI_STATUS Status;
UINT16 VendorID = 0;
UINT16 DeviceID = 0;
EFI_PCI_IO_PROTOCOL *PciIo;
// open PCI protocol
Status = gBS->OpenProtocol (
Controller,
&gEfiPciIoProtocolGuid,
(VOID **) &PciIo,
This->DriverBindingHandle,
Controller,
EFI_OPEN_PROTOCOL_BY_DRIVER
);
if (EFI_ERROR (Status)) {
return Status;
}
//Read deviceID and vendorID
PciIo->Pci.Read (
PciIo,
EfiPciIoWidthUint16,
0, // word 0,1 = VendorID
1,
&VendorID
);
PciIo->Pci.Read (
PciIo,
EfiPciIoWidthUint16,
2, // words 2,3 = DeviceID
1,
&DeviceID
);
gBS->CloseProtocol (
Controller,
&gEfiPciIoProtocolGuid,
This->DriverBindingHandle,
Controller
);
if (VendorID == my_vendor_id) {
return EFI_SUCCESS;
} else {
return EFI_UNSUPPORTED;
}
}
/**
Start this driver on ControllerHandle.
This service is called by the EFI boot service ConnectController(). In order
to make drivers as small as possible, there are a few calling restrictions for
this service. ConnectController() must follow these calling restrictions. If
any other agent wishes to call Start() it must also follow these calling
restrictions.
@param This Protocol instance pointer.
@param ControllerHandle Handle of device to bind driver to
@param RemainingDevicePath Optional parameter use to pick a specific child
device to start.
@retval EFI_SUCCESS This driver is added to ControllerHandle
@retval EFI_ALREADY_STARTED This driver is already running on ControllerHandle
@retval other This driver does not support this device
**/
EFI_STATUS
EFIAPI
SpDriverBindingStart (
IN EFI_DRIVER_BINDING_PROTOCOL *This,
IN EFI_HANDLE Controller,
IN EFI_DEVICE_PATH_PROTOCOL *RemainingDevicePath OPTIONAL
)
{
EFI_STATUS Status;
EFI_PCI_IO_PROTOCOL *PciIo;
UINTN NumPages;
VOID *HostAddr,*Mapping;
EFI_PHYSICAL_ADDRESS DeviceAddr;
UINTN NumberOfBytes,Length;
Print(L"INSIDE START Function\n");
Status = gBS->OpenProtocol (
Controller,
&gEfiPciIoProtocolGuid,
(VOID **) &PciIo,
This->DriverBindingHandle,
Controller,
EFI_OPEN_PROTOCOL_BY_DRIVER
);
if (EFI_ERROR (Status)) {
return Status;
}
// Enable PCI device.
Status = PciIo->Attributes(
PciIo,
EfiPciIoAttributeOperationEnable,
EFI_PCI_DEVICE_ENABLE,
NULL
);
if (EFI_ERROR (Status)) {
return Status;
}
//Length = 0x200000000; //8GB
//Length = 0x100000000; //4GB
Length = 0xC0000000; //3GB
//Length = 0x80000000; //2GB
//Length = 0x40000000; //1GB
//Length = 0x20000000; //512MB
//Length = 0x8000000; //128MB
//Length = 0x4000000; //64MB
NumPages = EFI_SIZE_TO_PAGES(Length);
Status = PciIo->AllocateBuffer(
PciIo,
AllocateAnyPages,
EfiBootServicesData,
NumPages,
&HostAddr,
0);
if (EFI_ERROR (Status)) {
Print(L"\n AllocateBuffer Failed::******\n");
return Status;
}
NumberOfBytes = Length;
Status= PciIo->Map ( PciIo,
EfiPciIoOperationBusMasterCommonBuffer,
HostAddr,
&NumberOfBytes,
&DeviceAddr,
&Mapping );
if (!EFI_ERROR (Status) && NumberOfBytes != Length) {
Print(L"Map Failed**************\n");
PciIo->Unmap (PciIo, Mapping);
Status = EFI_OUT_OF_RESOURCES;
}
if (EFI_ERROR (Status)) {
PciIo->FreeBuffer (
PciIo,
EFI_SIZE_TO_PAGES (Length),
HostAddr
);
return Status;
}
//
// Write the DMA start add
Print(L"Allocated successfully\n");
return EFI_SUCCESS;
}
/**
Stop this driver on ControllerHandle.
This service is called by the EFI boot service DisconnectController().
In order to make drivers as small as possible, there are a few calling
restrictions for this service. DisconnectController() must follow these
calling restrictions. If any other agent wishes to call Stop() it must
also follow these calling restrictions.
@param This Protocol instance pointer.
@param ControllerHandle Handle of device to stop driver on
@param NumberOfChildren Number of Handles in ChildHandleBuffer. If number of
children is zero stop the entire bus driver.
@param ChildHandleBuffer List of Child Handles to Stop.
@retval EFI_SUCCESS This driver is removed ControllerHandle
@retval other This driver was not removed from this device
**/
EFI_STATUS
EFIAPI
SpDriverBindingStop (
IN EFI_DRIVER_BINDING_PROTOCOL *This,
IN EFI_HANDLE Controller,
IN UINTN NumberOfChildren,
IN EFI_HANDLE *ChildHandleBuffer OPTIONAL
)
{
gBS->CloseProtocol (
Controller,
&gEfiPciIoProtocolGuid,
This->DriverBindingHandle,
Controller
);
return EFI_SUCCESS;
}
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their applications. This 200-page book is written by three acclaimed
leaders in the field. The early access version is available now.
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
_______________________________________________
edk2-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/edk2-devel
