Re: [edk2] BaseTools: Add MINGW tool chain

Subject: Re: [edk2] BaseTools: Add MINGW tool chain

From: "Scott Duplichan" <scott@notabs.org>

To: <edk2-devel@lists.sourceforge.net>

Date: 2014-10-15 09:47:47

Gao, Liming [mailto:liming.gao@intel.com] wrote:

]Scott:
]  Does MINGW base on GCC? If yes, is it similar to which version GCC? If I verify the code in MINGW tool chain, ]do I need to verify it in native GCC tool chain? 
]
]  For size issue, have you compared the size between native GCC tool chain and MINGW tool chain? 

Hello Liming,

Bad news and good news. The bad news is the code generated by
MINGW is much bigger than that of native linux gcc. The reason is
this gnu linker bug/limitation:
    https://sourceware.org/bugzilla/show_bug.cgi?id=11539
When the COFF object format used by MINGW is linked, the linker is
unable to pull individual functions from a library. Instead, the
whole library is linked in. The result is a lot of dead code in the
final image. Because the issue was reported years ago, it seems
unlikely it will be fixed any time soon.

The good news is that there is a solution for this problem that
solves most of the other problems with running gcc builds of EDK2
on Windows. Here are the problems with MINGW I know of:
1) Image is big due to dead library code in final image.
2) Default calling convention is different. This makes MINGW code
   generation differ from gcc-linux code generation.
3) MINGW sizeof long differs from gnu-linux size of long, resulting
   in more code generation differences.
4) Gcc linker plug-in for link time code generation is not yet
   available for Windows.

The first 3 problems can be solved by building a gcc tool chain that
runs on Windows, but targets Linux rather than Windows. This gcc
reports "Target: x86_64-linux-gnu", just like a native Linux gcc.
It generates ELF object format, so it is unaffected by ld bug 11539.
It uses the same calling conventions and LP64 data model as Linux,
so the code generation should match. Problem 4) is not solved, but
that is not an issue today because EDK2 doesn't yet use gcc link
time optimization.

I have documented how to build from this tool chain from scratch
on Windows 7. It is a Windows hosted gcc 4.9.1 targeting
x86_64-linux-gnu. It is a drop-in replacement for MINGW when used
as an EDK2 tool chain. It can be built as 32-bit executables or
64-bit executables. There is a problem with ar.exe in the 64-bit
build, so I am using the 32-bit build at the moment.

I will send an experimental patch as well as build instructions
and binaries in a couple of days. I think this will end up being
the preferred way to make gcc builds of EDK2 using Windows.

Thanks,
Scott
  
]  For Nasm issue in BaseLib, MINGW output is COFF format. So, NASM_FLAGS should be -f win32 and -f win64. I just ]try them and pass build. 
]
]  For tool_def.txt, on MAKE and RC path, could we directly use make and objcopy from MINGW like the native GCC ]tool chain? If so, MINGW is a standalone tool chain. 
]+*_MINGW_*_MAKE_PATH          = ENV(MS_MISC_BIN)\nmake.exe
]+*_MINGW_IA32_RC_PATH         = ENV(MS_MISC_BIN)\rc.exe
]
]Thanks
]Liming
-----Original Message-----
From: Scott Duplichan [mailto:scott@notabs.org] 
Sent: Saturday, October 04, 2014 11:42 AM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] BaseTools: Add MINGW tool chain

Nikolai Saoukh [mailto:nms@otdel-1.org] wrote:

]I was under impression that mingw-win64 can be used for X64 AND IA32 builds.

You are right. Thanks for pointing that out. The revised email and patch follows.
I also added the requirement for an environment variable pointing to Microsoft nmake.exe and rc.exe. Still needed is nasm.exe and iasl.exe, but these work the same as with Microsoft tool chains.
-- 

Here is a work in progress patch for those building X64 or IA32 targets from Windows. It allows building with gcc instead of a Microsoft compiler.
Building has been tested with Windows 7 x64 using MINGW archives from http://www.drangon.org/mingw/ or http://sourceforge.net/projects/mingw-w64-dgn/files/mingw-w64/

Unpack mingw-w64-bin-x86_64-20140425.7z and set environment variable
MINGW64 to its path. Also set environment variable MINGW32 to its path.
Set environment variable MS_MISC_BIN to a directory containing Microsoft nmake.exe and rc.exe. For example, the bin\x86 directory of DDK-7600:
http://www.microsoft.com/en-us/download/details.aspx?id=11800. 

Apply the patch then build using -t MINGW. It's that easy. No unix shell port is involved. Just build from a cmd.exe prompt as normal.
Because mingw requires no install or setup, your registry, path and other environment variables are unaffected.

The MINGW port of gcc brings C99 support to those using Windows to develop EDK2 code. Wouldn't it be nice if one day UEFI could drop support for non-C99 compilers so that C99 features could be used in the code? There is actually a lot of work to be done before gcc could replace Microsoft tools. One problem is that some UEFI source level debuggers understand only Microsoft debug information. Another problem is that right now gcc is producing bigger code. Getting gcc link time optimization working may solve this problem. Build speed with MINGW is not quite as fast as with Microsoft tools, but it is reasonable.
For example, a test of 18 builds that takes 523 seconds using VS2008 takes 694 seconds using MINGW.

Index: BaseTools/Conf/tools_def.template ===================================================================
--- BaseTools/Conf/tools_def.template	(revision 16195)
+++ BaseTools/Conf/tools_def.template	(working copy)
@@ -4815,6 +4815,103 @@
 
 ####################################################################################
 #
+# Mingw GCC And Intel ACPI Compiler
+#
+####################################################################################
+#   MINGW        - MINGW GCC
+#   ASL           - Intel ACPI Source Language Compiler (iasl.exe)
+*_MINGW_*_*_FAMILY           = GCC
+
+DEFINE MINGW_BINIA32         = ENV(MINGW32)\bin\
+DEFINE MINGW_BINX64          = ENV(MINGW64)\bin\
+
+*_MINGW_*_MAKE_PATH          = ENV(MS_MISC_BIN)\nmake.exe
+*_MINGW_*_ASL_PATH           = DEF(DEFAULT_WIN_ASL_BIN)
+*_MINGW_IA32_DLINK_FLAGS     = DEF(GCC_IA32_X64_DLINK_COMMON) -m i386pe  --entry _$(IMAGE_ENTRY_POINT) --file-alignment 0x20 --section-alignment 0x20 -Map $(DEST_DIR_DEBUG)/$(BASE_NAME).map --image-base=0 --strip-all
+*_MINGW_X64_DLINK_FLAGS      = DEF(GCC_IA32_X64_DLINK_COMMON) -m i386pep --entry  $(IMAGE_ENTRY_POINT) --file-alignment 0x20 --section-alignment 0x20 -Map $(DEST_DIR_DEBUG)/$(BASE_NAME).map --image-base=0 --strip-all
+*_MINGW_IA32_ASLDLINK_FLAGS  = DEF(GCC_IA32_X64_DLINK_COMMON) -m i386pe  --entry _ReferenceAcpiTable -u $(IMAGE_ENTRY_POINT) --strip-all
+*_MINGW_X64_ASLDLINK_FLAGS   = DEF(GCC_IA32_X64_DLINK_COMMON) -m i386pep --entry  ReferenceAcpiTable -u $(IMAGE_ENTRY_POINT) --strip-all
+
+*_MINGW_*_MAKE_FLAGS         = /nologo
+*_MINGW_*_ASL_FLAGS          = DEF(DEFAULT_WIN_ASL_FLAGS)
+*_MINGW_*_ASL_OUTFLAGS       = DEF(DEFAULT_WIN_ASL_OUTFLAGS)
+
+##################
+# IA32 definitions
+##################
+*_MINGW_IA32_CC_PATH         = DEF(MINGW_BINIA32)gcc
+*_MINGW_IA32_SLINK_PATH      = DEF(MINGW_BINIA32)ar
+*_MINGW_IA32_DLINK_PATH      = DEF(MINGW_BINIA32)ld
+*_MINGW_IA32_ASM_PATH        = DEF(MINGW_BINIA32)gcc
+*_MINGW_IA32_PP_PATH         = DEF(MINGW_BINIA32)gcc
+*_MINGW_IA32_APP_PATH        = DEF(MINGW_BINIA32)gcc
+*_MINGW_IA32_VFRPP_PATH      = DEF(MINGW_BINIA32)gcc
+*_MINGW_IA32_ASLCC_PATH      = DEF(MINGW_BINIA32)gcc
+*_MINGW_IA32_ASLPP_PATH      = DEF(MINGW_BINIA32)gcc
+*_MINGW_IA32_ASLDLINK_PATH   = DEF(MINGW_BINIA32)ld
+*_MINGW_IA32_RC_PATH         = ENV(MS_MISC_BIN)\rc.exe
+*_MINGW_IA32_OBJCOPY_PATH    = DEF(MINGW_BINIA32)objcopy
+*_MINGW_IA32_SYMRENAME_PATH  = DEF(MINGW_BINIA32)objcopy
+
+*_MINGW_IA32_CC_FLAGS        = DEF(GCC_ALL_CC_FLAGS) -m32 -Os -malign-double -mno-stack-arg-probe -fomit-frame-pointer -Wno-error
+*_MINGW_IA32_PP_FLAGS        = DEF(GCC_PP_FLAGS) -m32
+*_MINGW_IA32_ASM_FLAGS       = DEF(GCC_ASM_FLAGS) -m32
+*_MINGW_IA32_ASLPP_FLAGS     = DEF(GCC_ASLPP_FLAGS) -m32
+*_MINGW_IA32_ASLCC_FLAGS     = DEF(GCC_ASLCC_FLAGS) -m32
+*_MINGW_IA32_VFRPP_FLAGS     = DEF(GCC_VFRPP_FLAGS) -m32
+*_MINGW_IA32_APP_FLAGS       = -m32
+*_MINGW_IA32_OBJCOPY_FLAGS   = --remove-section .idata --remove-section .edata
+*_MINGW_IA32_SYMRENAME_FLAGS = --remove-section .idata --remove-section .edata
+*_MINGW_IA32_RC_FLAGS        = DEF(GCC_IA32_RC_FLAGS)
+*_MINGW_IA32_NASM_FLAGS      = -f elf32
+
+##################
+# X64 definitions
+##################
+*_MINGW_X64_CC_PATH          = DEF(MINGW_BINX64)gcc
+*_MINGW_X64_SLINK_PATH       = DEF(MINGW_BINX64)ar
+*_MINGW_X64_DLINK_PATH       = DEF(MINGW_BINX64)ld
+*_MINGW_X64_ASM_PATH         = DEF(MINGW_BINX64)gcc
+*_MINGW_X64_PP_PATH          = DEF(MINGW_BINX64)gcc
+*_MINGW_X64_APP_PATH         = DEF(MINGW_BINX64)gcc
+*_MINGW_X64_VFRPP_PATH       = DEF(MINGW_BINX64)gcc
+*_MINGW_X64_ASLCC_PATH       = DEF(MINGW_BINX64)gcc
+*_MINGW_X64_ASLPP_PATH       = DEF(MINGW_BINX64)gcc
+*_MINGW_X64_ASLDLINK_PATH    = DEF(MINGW_BINX64)ld
+*_MINGW_IA32_RC_PATH         = ENV(MS_MISC_BIN)\rc.exe
+*_MINGW_X64_OBJCOPY_PATH     = DEF(MINGW_BINX64)objcopy
+*_MINGW_X64_SYMRENAME_PATH   = rem
+
+*_MINGW_X64_CC_FLAGS         = DEF(GCC_X64_CC_FLAGS) -m64 -Wno-error
+*_MINGW_X64_PP_FLAGS         = DEF(GCC_PP_FLAGS) -m64
+*_MINGW_X64_ASM_FLAGS        = DEF(GCC_ASM_FLAGS) -m64
+*_MINGW_X64_ASLPP_FLAGS      = DEF(GCC_ASLPP_FLAGS) -m64
+*_MINGW_X64_ASLCC_FLAGS      = DEF(GCC_ASLCC_FLAGS) -m64
+*_MINGW_X64_VFRPP_FLAGS      = DEF(GCC_VFRPP_FLAGS) -m64
+*_MINGW_X64_APP_FLAGS        = -m64
+*_MINGW_X64_OBJCOPY_FLAGS    = --remove-section .pdata --remove-section .xdata
+*_MINGW_X64_SYMRENAME_FLAGS  = Symbol renaming not needed for
+*_MINGW_X64_RC_FLAGS         = DEF(GCC_X64_RC_FLAGS)
+*_MINGW_X64_NASM_FLAGS       = -f elf64
+
+##################
+# EBC definitions
+##################
+*_MINGW_EBC_*_FAMILY      = INTEL
+
+*_MINGW_EBC_PP_PATH       = DEF(EBC_BIN)\iec.exe
+*_MINGW_EBC_CC_PATH       = DEF(EBC_BIN)\iec.exe
+*_MINGW_EBC_DLINK_PATH    = DEF(EBC_BIN)\link.exe
+*_MINGW_EBC_SLINK_PATH    = DEF(EBC_BIN)\link.exe
+*_MINGW_EBC_VFRPP_PATH    = DEF(EBC_BIN)\iec.exe
+*_MINGW_EBC_RC_PATH       = ENV(MS_MISC_BIN)\rc.exe
+
+*_MINGW_EBC_CC_FLAGS      = /nologo /c /WX /W3 /FIAutoGen.h /D$(MODULE_ENTRY_POINT)=$(ARCH_ENTRY_POINT)
+*_MINGW_EBC_SLINK_FLAGS   = /lib /NOLOGO /MACHINE:EBC
+*_MINGW_EBC_DLINK_FLAGS   = "C:\Program Files\Intel\EBC\Lib\EbcLib.lib" /NOLOGO /NODEFAULTLIB /MACHINE:EBC /OPT:REF /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /ALIGN:32 /DRIVER
+
+#######################################################################
+#############
+#
 # Elf GCC - This configuration is used to compile on Linux boxes to produce elf
 #           binaries.
 #
Index: BaseTools/Source/Python/AutoGen/GenMake.py
===================================================================
--- BaseTools/Source/Python/AutoGen/GenMake.py	(revision 16195)
+++ BaseTools/Source/Python/AutoGen/GenMake.py	(working copy)
@@ -679,7 +679,11 @@
                     NewFile = self.PlaceMacro(str(F), self.Macros)
                     # In order to use file list macro as dependency
                     if T.GenListFile:
-                        self.ListFileMacros[T.ListFileMacro].append(str(F))
+                        # gnu tools need forward slash path separater, even on Windows
+                        if self.PlatformInfo.ToolChainFamily in ('GCC'):
+                            self.ListFileMacros[T.ListFileMacro].append(str(F).replace ('\\', '/'))
+                        else:
+                            
+ self.ListFileMacros[T.ListFileMacro].append(str(F))
                         self.FileListMacros[T.FileListMacro].append(NewFile)
                     elif T.GenFileListMacro:
                         self.FileListMacros[T.FileListMacro].append(NewFile)
Index: MdePkg/Library/BaseLib/BaseLib.inf
===================================================================
--- MdePkg/Library/BaseLib/BaseLib.inf	(revision 16195)
+++ MdePkg/Library/BaseLib/BaseLib.inf	(working copy)
@@ -252,7 +252,7 @@
   Ia32/DisableCache.asm | INTEL
 
   Ia32/GccInline.c | GCC
-  Ia32/Thunk16.nasm | GCC 
+  Ia32/Thunk16.S | GCC
   Ia32/EnableDisableInterrupts.S | GCC 
   Ia32/EnablePaging64.S | GCC 
   Ia32/DisablePaging32.S | GCC
@@ -405,7 +405,7 @@
   X86DisablePaging64.c
   X86DisablePaging32.c
   X64/GccInline.c | GCC
-  X64/Thunk16.nasm | GCC 
+  X64/Thunk16.S | GCC
   X64/SwitchStack.S | GCC 
   X64/SetJump.S | GCC 
   X64/LongJump.S | GCC
Index: StdLib/EfiSocketLib/EfiSocketLib.inf
===================================================================
--- StdLib/EfiSocketLib/EfiSocketLib.inf	(revision 16195)
+++ StdLib/EfiSocketLib/EfiSocketLib.inf	(working copy)
@@ -68,3 +68,6 @@
   gEfiUdp6ServiceBindingProtocolGuid
   gEfiSocketProtocolGuid
   gEfiSocketServiceBindingProtocolGuid
+
+[BuildOptions]
+  GCC:*_MINGW_*_CC_FLAGS = -D__attribute__(weak)=
Index: StdLib/Include/X64/machine/limits.h
===================================================================
--- StdLib/Include/X64/machine/limits.h	(revision 16195)
+++ StdLib/Include/X64/machine/limits.h	(working copy)
@@ -2,7 +2,7 @@
 
 #define __POINTER_BIT   64
 
-#if defined(__GNUC__)
+#if defined(__GNUC__) && !defined(_WIN32)
 #if __GNUC_PREREQ__(4,4)
   #define __LONG_BIT      64

--
Thanks,
Scott
------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/edk2-devel


------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/edk2-devel