Re: [edk2] [PATCH RFC] MdeModulePkg/PartitionDxe: Add UDF/ECMA-167 filesystem support

Subject: Re: [edk2] [PATCH RFC] MdeModulePkg/PartitionDxe: Add UDF/ECMA-167 filesystem support

From: Andrew Fish <afish@apple.com>

To: edk2-devel@lists.sourceforge.net

Date: 2014-10-30 06:23:48


> On Oct 29, 2014, at 10:08 PM, Tian, Feng  wrote:
> 
> Thanks for your great contribution, Paulo.
> 
> For the device path issue, my personal idea is vendor device path may not be a good idea and I prefer to introduce a new UDF device path node. But let's us wait other's comments at first.
> 

What information would need to be present in an UDF device path node?

Thanks,

Andrew Fish

> Thanks
> Feng
> 
> -----Original Message-----
> From: Paulo Alcantara [mailto:pcacjr@gmail.com] 
> Sent: Thursday, October 30, 2014 10:53
> To: edk2-devel@lists.sourceforge.net
> Subject: [edk2] [PATCH RFC] MdeModulePkg/PartitionDxe: Add UDF/ECMA-167 filesystem support
> 
> This patch adds support to finding UDF/ECMA-167 volumes on disks by looking at the UDF Standard Identifiers (BEA01, NSR02/NSR03, TEA01) and the Anchor Volume Descriptor Pointer (AVDP) at LBAs 256, N - 256 or N.
> 
> A new PcdsFeatureFlags (PcdUdfFileSystemSupport) has been added to enable/disable such support -- it's disabled by default.
> 
> I'm currently using VENDOR_DEVICE_PATH for UDF since there isn't any defined for it yet. Should we create one specific for it? Or should we keep using VENDOR_DEVICE_PATH?
> 
> The tests were run under QEMU + OVMF:
> $ qemu-kvm -m 1024 -L . -hda fat:hdd \
>  file=/dev/sdb,if=virtio,media=disk,cache=writeback \
>  -net none
> 
> Where /dev/sdb is a external HDD previously formatted as an UDF filesystem through the following command:
> 
> $ mkudffs -b 512 --media-type=hd /dev/sdb
> 
> Besides, I've made a good progress on the filesystem driver (UdfDxe). It now supports symlinks, handle inodes which are either File Entry or Extended File Entry, handle both long and short allocation descriptors, as well as the usual
> things: open, read and list files. I'm not going to send it now because I need to test it more, rework the commit messages and a few cleanups which are always good :-)
> 
> My development branches:
> git://git.zytor.com/users/pcacjr/edk2.git partitiondxe (PartitionDxe) git://git.zytor.com/users/pcacjr/edk2.git udf-next (UdfDxe)
> 
> Thanks,
> 
>    -pcacjr
> 
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Paulo Alcantara 
> ---
> MdeModulePkg/MdeModulePkg.dec                      |   6 +
> .../Universal/Disk/PartitionDxe/Partition.c        |   1 +
> .../Universal/Disk/PartitionDxe/Partition.h        |  29 +++
> .../Universal/Disk/PartitionDxe/PartitionDxe.inf   |   9 +-
> MdeModulePkg/Universal/Disk/PartitionDxe/Udf.c     | 277 +++++++++++++++++++++
> MdePkg/Include/IndustryStandard/Udf.h              |  88 +++++++
> 6 files changed, 409 insertions(+), 1 deletion(-)  create mode 100644 MdeModulePkg/Universal/Disk/PartitionDxe/Udf.c
> create mode 100644 MdePkg/Include/IndustryStandard/Udf.h
> 
> diff --git a/MdeModulePkg/MdeModulePkg.dec b/MdeModulePkg/MdeModulePkg.dec index 60fb209..e320e1b 100644
> --- a/MdeModulePkg/MdeModulePkg.dec
> +++ b/MdeModulePkg/MdeModulePkg.dec
> @@ -597,6 +597,12 @@
>   # @Prompt Enable S3 performance data support.
>   gEfiMdeModulePkgTokenSpaceGuid.PcdFirmwarePerformanceDataTableS3Support|TRUE|BOOLEAN|0x00010064
> 
> +  ## Indicates if UDF/ECMA-167 filesystem will be supported.

> + # TRUE - UDF/ECMA-167 filesystem will be supported.
> + # FALSE - UDF/ECMA-167 filesystem will not be supported.
> + # @Prompt Enable UDF/ECMA-167 filesystem support. > + > + gEfiMdeModulePkgTokenSpaceGuid.PcdUdfFileSystemSupport|FALSE|BOOLEAN|0 > + x00010069 > + > [PcdsFeatureFlag.IA32, PcdsFeatureFlag.X64] > ## Indicates if DxeIpl should switch to long mode to enter DXE phase. > # It is assumed that 64-bit DxeCore is built in firmware if it is true; otherwise 32-bit DxeCore diff --git a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c > index 28639b0..27e460f 100644 > --- a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c > +++ b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c > @@ -45,6 +45,7 @@ PARTITION_DETECT_ROUTINE mPartitionDetectRoutineTable[] = { > PartitionInstallGptChildHandles, > PartitionInstallElToritoChildHandles, > PartitionInstallMbrChildHandles, > + PartitionInstallUdfChildHandles, > NULL > }; > > diff --git a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.h b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.h > index 06470f6..26107f0 100644 > --- a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.h > +++ b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.h > @@ -38,6 +38,7 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. > > #include > #include > +#include > > > // > @@ -428,6 +429,34 @@ PartitionInstallMbrChildHandles ( > IN EFI_DEVICE_PATH_PROTOCOL *DevicePath > ); > > +/** > + Install child handles if the Handle supports UDF/ECMA-167 volume format. > + > + @param[in] This Calling context. > + @param[in] Handle Parent Handle. > + @param[in] DiskIo Parent DiskIo interface. > + @param[in] DiskIo2 Parent DiskIo2 interface. > + @param[in] BlockIo Parent BlockIo interface. > + @param[in] BlockIo2 Parent BlockIo2 interface. > + @param[in] DevicePath Parent Device Path > + > + > + @retval EFI_SUCCESS Child handle(s) was added. > + @retval EFI_MEDIA_CHANGED Media changed Detected. > + @retval other no child handle was added. > + > +**/ > +EFI_STATUS > +PartitionInstallUdfChildHandles ( > + IN EFI_DRIVER_BINDING_PROTOCOL *This, > + IN EFI_HANDLE Handle, > + IN EFI_DISK_IO_PROTOCOL *DiskIo, > + IN EFI_DISK_IO2_PROTOCOL *DiskIo2, > + IN EFI_BLOCK_IO_PROTOCOL *BlockIo, > + IN EFI_BLOCK_IO2_PROTOCOL *BlockIo2, > + IN EFI_DEVICE_PATH_PROTOCOL *DevicePath > + ); > + > typedef > EFI_STATUS > (*PARTITION_DETECT_ROUTINE) ( > diff --git a/MdeModulePkg/Universal/Disk/PartitionDxe/PartitionDxe.inf b/MdeModulePkg/Universal/Disk/PartitionDxe/PartitionDxe.inf > index 6806263..1ce9b44 100644 > --- a/MdeModulePkg/Universal/Disk/PartitionDxe/PartitionDxe.inf > +++ b/MdeModulePkg/Universal/Disk/PartitionDxe/PartitionDxe.inf > @@ -46,13 +46,14 @@ > Mbr.c > Gpt.c > ElTorito.c > + Udf.c > Partition.c > Partition.h > > > [Packages] > MdePkg/MdePkg.dec > - > + MdeModulePkg/MdeModulePkg.dec > > [LibraryClasses] > DevicePathLib > @@ -63,6 +64,7 @@ > BaseLib > UefiDriverEntryPoint > DebugLib > + PcdLib > > > [Guids] > @@ -85,5 +87,10 @@ > gEfiDiskIoProtocolGuid ## TO_START > gEfiDiskIo2ProtocolGuid ## TO_START > > + > +[FeaturePcd] > + gEfiMdeModulePkgTokenSpaceGuid.PcdUdfFileSystemSupport ## CONSUMES > + > + > [UserExtensions.TianoCore."ExtraFiles"] > PartitionDxeExtra.uni > diff --git a/MdeModulePkg/Universal/Disk/PartitionDxe/Udf.c b/MdeModulePkg/Universal/Disk/PartitionDxe/Udf.c > new file mode 100644 > index 0000000..4c2fc4f > --- /dev/null > +++ b/MdeModulePkg/Universal/Disk/PartitionDxe/Udf.c > @@ -0,0 +1,277 @@ > +/** @file > + Decode an UDF/ECMA-167 formatted medium > + > +Copyright (c) 2014 Paulo Alcantara
This program > +and the accompanying materials are licensed and made available under > +the terms and conditions of the BSD License which accompanies this > +distribution. The full text of the license may be found at > +http://opensource.org/licenses/bsd-license.php > + > +THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, > +WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. > + > +**/ > + > + > +#include "Partition.h" > + > + > +EFI_GUID gUdfVolumeSignatureGuid = { > + 0xC5BD4D42, 0x1A76, 0x4996, > + { 0x89, 0x56, 0x73, 0xCD, 0xA3, 0x26, 0xCD, 0x0A } }; > + > +UDF_STANDARD_IDENTIFIER > +gUdfStandardIdentifiers[NR_STANDARD_IDENTIFIERS] = { > + { { 'B', 'E', 'A', '0', '1' } }, > + { { 'N', 'S', 'R', '0', '2' } }, > + { { 'N', 'S', 'R', '0', '3' } }, > + { { 'T', 'E', 'A', '0', '1' } }, > +}; > + > +typedef struct { > + VENDOR_DEVICE_PATH DevicePath; > + EFI_DEVICE_PATH_PROTOCOL End; > +} UDF_DEVICE_PATH; > + > +// > +// C5BD4D42-1A76-4996-8956-73CDA326CD0A > +// > +#define EFI_UDF_DEVICE_PATH_GUID \ > + { 0xC5BD4D42, 0x1A76, 0x4996, \ > + { 0x89, 0x56, 0x73, 0xCD, 0xA3, 0x26, 0xCD, 0x0A } \ > + } > + > +UDF_DEVICE_PATH gUdfDevicePath = { > + { { MEDIA_DEVICE_PATH, MEDIA_VENDOR_DP, > + { sizeof (VENDOR_DEVICE_PATH), 0 } }, > + EFI_UDF_DEVICE_PATH_GUID > + }, > + { END_DEVICE_PATH_TYPE, END_ENTIRE_DEVICE_PATH_SUBTYPE, > + { sizeof (EFI_DEVICE_PATH_PROTOCOL), 0 } > + } > +}; > + > +EFI_STATUS > +FindAnchorVolumeDescriptorPointer ( > + IN EFI_BLOCK_IO_PROTOCOL *BlockIo, > + IN EFI_DISK_IO_PROTOCOL *DiskIo, > + OUT UDF_ANCHOR_VOLUME_DESCRIPTOR_POINTER *AnchorPoint > + ) > +{ > + EFI_STATUS Status; > + UINT32 BlockSize; > + EFI_LBA EndLBA; > + > + BlockSize = BlockIo->Media->BlockSize; > + EndLBA = BlockIo->Media->LastBlock; > + Status = DiskIo->ReadDisk ( > + DiskIo, > + BlockIo->Media->MediaId, > + MultU64x32 (0x100ULL, BlockSize), > + sizeof (UDF_ANCHOR_VOLUME_DESCRIPTOR_POINTER), > + (VOID *) AnchorPoint > + ); > + if (EFI_ERROR (Status)) { > + goto Exit; > + } > + Status = EFI_VOLUME_CORRUPTED; > + if (IS_AVDP (AnchorPoint)) { > + Status = EFI_SUCCESS; > + goto Exit; > + } > + > + Status = DiskIo->ReadDisk ( > + DiskIo, > + BlockIo->Media->MediaId, > + MultU64x32 (EndLBA - 0x100ULL, BlockSize), > + sizeof (UDF_ANCHOR_VOLUME_DESCRIPTOR_POINTER), > + (VOID *) AnchorPoint > + ); > + if (EFI_ERROR (Status)) { > + goto Exit; > + } > + if (IS_AVDP (AnchorPoint)) { > + Status = EFI_SUCCESS; > + goto Exit; > + } > + > + Status = DiskIo->ReadDisk ( > + DiskIo, > + BlockIo->Media->MediaId, > + MultU64x32 (EndLBA, BlockSize), > + sizeof (UDF_ANCHOR_VOLUME_DESCRIPTOR_POINTER), > + (VOID *) AnchorPoint > + ); > + if (EFI_ERROR (Status)) { > + goto Exit; > + } > + if (IS_AVDP (AnchorPoint)) { > + Status = EFI_SUCCESS; > + } > + > +Exit: > + return Status; > +} > + > +EFI_STATUS > +IsSupportedUdfVolume ( > + IN EFI_BLOCK_IO_PROTOCOL *BlockIo, > + IN EFI_DISK_IO_PROTOCOL *DiskIo, > + OUT BOOLEAN *Supported > + ) > +{ > + EFI_STATUS Status; > + UDF_NSR_DESCRIPTOR NsrDescriptor; > + UDF_ANCHOR_VOLUME_DESCRIPTOR_POINTER AnchorPoint; > + > + *Supported = FALSE; > + // > + // Start Volume Recognition Sequence > + // > + Status = DiskIo->ReadDisk ( > + DiskIo, > + BlockIo->Media->MediaId, > + BEA_DESCRIPTOR_LSN_OFFSET, > + sizeof (UDF_NSR_DESCRIPTOR), > + (VOID *) &NsrDescriptor > + ); > + if (EFI_ERROR (Status)) { > + goto Exit; > + } > + if (CompareMem ( > + (VOID *) &NsrDescriptor.StandardIdentifier, > + (VOID *) &gUdfStandardIdentifiers[BEA_IDENTIFIER], > + UDF_STANDARD_IDENTIFIER_LENGTH > + ) > + ) { > + goto Exit; > + } > + Status = DiskIo->ReadDisk ( > + DiskIo, > + BlockIo->Media->MediaId, > + BEA_DESCRIPTOR_LSN_OFFSET + LOGICAL_SECTOR_SIZE, > + sizeof (UDF_NSR_DESCRIPTOR), > + (VOID *) &NsrDescriptor > + ); > + if (EFI_ERROR (Status)) { > + goto Exit; > + } > + if (CompareMem ( > + (VOID *) &NsrDescriptor.StandardIdentifier, > + (VOID *) &gUdfStandardIdentifiers[VSD_IDENTIFIER_0], > + UDF_STANDARD_IDENTIFIER_LENGTH > + ) > + && > + CompareMem ( > + (VOID *) &NsrDescriptor.StandardIdentifier, > + (VOID *) &gUdfStandardIdentifiers[VSD_IDENTIFIER_1], > + UDF_STANDARD_IDENTIFIER_LENGTH > + ) > + ) { > + goto Exit; > + } > + Status = DiskIo->ReadDisk ( > + DiskIo, > + BlockIo->Media->MediaId, > + BEA_DESCRIPTOR_LSN_OFFSET + (LOGICAL_SECTOR_SIZE << 1), > + sizeof (UDF_NSR_DESCRIPTOR), > + (VOID *)&NsrDescriptor > + ); > + if (EFI_ERROR (Status)) { > + goto Exit; > + } > + if (CompareMem ( > + (VOID *) &NsrDescriptor.StandardIdentifier, > + (VOID *) &gUdfStandardIdentifiers[TEA_IDENTIFIER], > + UDF_STANDARD_IDENTIFIER_LENGTH > + ) > + ) { > + goto Exit; > + } > + > + Status = FindAnchorVolumeDescriptorPointer (BlockIo, DiskIo, > + &AnchorPoint); if (!EFI_ERROR (Status)) { > + *Supported = TRUE; > + } > + > +Exit: > + return Status; > +} > + > +/** > + Install child handles if the Handle supports UDF/ECMA-167 volume format. > + > + @param[in] This Calling context. > + @param[in] Handle Parent Handle. > + @param[in] DiskIo Parent DiskIo interface. > + @param[in] DiskIo2 Parent DiskIo2 interface. > + @param[in] BlockIo Parent BlockIo interface. > + @param[in] BlockIo2 Parent BlockIo2 interface. > + @param[in] DevicePath Parent Device Path > + > + > + @retval EFI_SUCCESS Child handle(s) was added. > + @retval EFI_MEDIA_CHANGED Media changed Detected. > + @retval other no child handle was added. > + > +**/ > +EFI_STATUS > +PartitionInstallUdfChildHandles ( > + IN EFI_DRIVER_BINDING_PROTOCOL *This, > + IN EFI_HANDLE Handle, > + IN EFI_DISK_IO_PROTOCOL *DiskIo, > + IN EFI_DISK_IO2_PROTOCOL *DiskIo2, > + IN EFI_BLOCK_IO_PROTOCOL *BlockIo, > + IN EFI_BLOCK_IO2_PROTOCOL *BlockIo2, > + IN EFI_DEVICE_PATH_PROTOCOL *DevicePath > + ) > +{ > + EFI_STATUS Status; > + BOOLEAN Supported; > + EFI_DEVICE_PATH_PROTOCOL *LastDevicePathNode; > + EFI_DEVICE_PATH_PROTOCOL *DevicePathNode; > + EFI_GUID *VendorDevGuid; > + EFI_GUID UdfDevGuid = EFI_UDF_DEVICE_PATH_GUID; > + > + if (!FeaturePcdGet (PcdUdfFileSystemSupport)) { > + return EFI_NOT_FOUND; > + } > + > + Status = IsSupportedUdfVolume (BlockIo, DiskIo, &Supported); if > + (EFI_ERROR (Status) || !Supported) { > + return EFI_NOT_FOUND; > + } > + > + LastDevicePathNode = NULL; > + DevicePathNode = DevicePath; > + while (!IsDevicePathEnd (DevicePathNode)) { > + LastDevicePathNode = DevicePathNode; > + DevicePathNode = NextDevicePathNode (DevicePathNode); > + } > + if (LastDevicePathNode) { > + VendorDevGuid = (EFI_GUID *) ((UINT8 *) LastDevicePathNode + OFFSET_OF (VENDOR_DEVICE_PATH, Guid)); > + if (DevicePathSubType (LastDevicePathNode) == MEDIA_VENDOR_DP && > + CompareGuid (VendorDevGuid, &UdfDevGuid)) { > + return EFI_NOT_FOUND; > + } > + } > + > + Status = PartitionInstallChildHandle ( > + This, > + Handle, > + DiskIo, > + DiskIo2, > + BlockIo, > + BlockIo2, > + DevicePath, > + (EFI_DEVICE_PATH_PROTOCOL *) &gUdfDevicePath, > + 0, > + BlockIo->Media->LastBlock, > + BlockIo->Media->BlockSize, > + FALSE > + ); > + if (!EFI_ERROR (Status)) { > + Status = EFI_NOT_FOUND; > + } > + return Status; > +} > diff --git a/MdePkg/Include/IndustryStandard/Udf.h b/MdePkg/Include/IndustryStandard/Udf.h > new file mode 100644 > index 0000000..8b41d05 > --- /dev/null > +++ b/MdePkg/Include/IndustryStandard/Udf.h > @@ -0,0 +1,88 @@ > +/** @file > + UDF/ECMA-167 Volume and File Structure Format Definition. > + > +Copyright (c) 2014 Paulo Alcantara
This program > +and the accompanying materials are licensed and made available under > +the terms and conditions of the BSD License which accompanies this > +distribution. The full text of the license may be found at > +http://opensource.org/licenses/bsd-license.php > + > +THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, > +WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. > + > +**/ > + > +#ifndef _UDF_H_ > +#define _UDF_H_ > + > +// > +// As specified in ECMA-167 specification, the logical sector size > +shall be // 2048 bytes. > +// > +#define LOGICAL_SECTOR_SIZE ((UINT64) 0x0000000000000800ULL) > +#define FIRST_ANCHOR_POINT_LSN ((UINT64) 0x0000000000000100ULL) > +#define BEA_DESCRIPTOR_LSN_OFFSET ((UINT64) 0x0000000000008000ULL) > +#define UDF_STANDARD_IDENTIFIER_LENGTH 5 > + > +#define _GET_TAG_ID(_Pointer) \ > + (((UDF_DESCRIPTOR_TAG *) (_Pointer))->TagIdentifier) > + > +#define IS_PVD(_Pointer) \ > + ((BOOLEAN)(_GET_TAG_ID (_Pointer) == 1)) #define IS_AVDP(_Pointer) \ > + ((BOOLEAN)(_GET_TAG_ID (_Pointer) == 2)) > + > +typedef struct { > + UINT8 StandardIdentifier[UDF_STANDARD_IDENTIFIER_LENGTH]; > +} UDF_STANDARD_IDENTIFIER; > + > +enum { > + BEA_IDENTIFIER, > + VSD_IDENTIFIER_0, > + VSD_IDENTIFIER_1, > + TEA_IDENTIFIER, > + NR_STANDARD_IDENTIFIERS, > +}; > + > +#pragma pack(1) > + > +typedef struct { > + UINT16 TagIdentifier; > + UINT16 DescriptorVersion; > + UINT8 TagChecksum; > + UINT8 Reserved; > + UINT16 TagSerialNumber; // Ignored. Intended for disaster recovery. > + UINT16 DescriptorCRC; > + UINT16 DescriptorCRCLength; > + UINT32 TagLocation; > +} UDF_DESCRIPTOR_TAG; > + > +// > +// ECMA 167 3/7.1 > +// > +typedef struct { > + UINT32 ExtentLength; > + UINT32 ExtentLocation; > +} UDF_EXTENT_AD; > + > +// > +// ECMA 167 3/9.1 > +// > +typedef struct { > + UINT8 StructureType; > + UINT8 StandardIdentifier[5]; > + UINT8 StructureVersion; > + UINT8 Reserved; > + UINT8 StructureData[2040]; > +} UDF_NSR_DESCRIPTOR; > + > +typedef struct { > + UDF_DESCRIPTOR_TAG DescriptorTag; > + UDF_EXTENT_AD MainVolumeDescriptorSequenceExtent; > + UDF_EXTENT_AD ReserveVolumeDescriptorSequenceExtent; > + UINT8 Reserved[480]; > +} UDF_ANCHOR_VOLUME_DESCRIPTOR_POINTER; > + > +#pragma pack() > + > +#endif // _UDF_H_ > -- > 1.9.3 > > > ------------------------------------------------------------------------------ > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/edk2-devel > > ------------------------------------------------------------------------------ > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/edk2-devel ------------------------------------------------------------------------------ _______________________________________________ edk2-devel mailing list edk2-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/edk2-devel