Re: [edk2] Shell Piping - set time via text file

Subject: Re: [edk2] Shell Piping - set time via text file

From: Andrew Fish <afish@apple.com>

To: edk2-devel@lists.sourceforge.net

Date: 2011-06-16 17:33:24


On Jun 16, 2011, at 9:22 AM, Carsey, Jaben wrote:

I was just thinking of pre-parsing environment variables and chopping off everything starting with the first EITHER CR or LF.  That should fix all of the different OSs you mention.
 
What do you think?

I think we either need to truncate starting at CR or LF or fully support multi line environment variables. 

Is there any way to embedded escape characters in a command line string? Given the parser will eat the CR, the only way a LF or CR could end up in a string from the command line would be an escape sequence. If it is not possible from the command line then truncating seems like the right approach. 

Thanks,

Andrew

 
-Jaben
 
From: Andrew Fish [mailto:afish@apple.com] 
Sent: Wednesday, June 15, 2011 4:12 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file
 
Jaben,
 
This is a much more complex issue that it appears on the surface. 
 
For one thing files can contain different line endings depending on what OS they come from (from Newline Wiki):

  CR+LFDEC TOPS-10RT-11 and most other early non-Unix and non-IBM OSes, CP/MMP/MDOS (MS-DOSPC-DOS, etc.), Atari TOSOS/2Microsoft WindowsSymbian OS,Palm OS

  LF+CRAcorn BBC spooled text output.

  CR:    Commodore 8-bit machines, Acorn BBCTRS-80Apple II familyMac OS up to version 9 and OS-9

  LF:    MulticsUnix and Unix-like systems (GNU/LinuxAIXXenixMac OS XFreeBSD, etc.), BeOSAmigaRISC OS, and others. However, in tty 'raw mode', CR+LF is used for output and CR is used for input.

 
Then you have the issue of UEFI not really being symmetric on input vs. output.  All the CR/LF stuff I mentioned was for output. On input it is really just CR. 
 
Lastly you need to throw in the fact that the input is being parsed, via EFI_SIMPLE_TEXT_INPUT_PROTOCOL.ReadKeyStroke() a single key press at a time. So for example the code in the shell that is processing a command line sees the CR and uses that to know to end the command line, but does not include it in the command line. 
 
So to make it work properly you need to make sure that you redirect via EFI_SIMPLE_TEXT_INPUT_PROTOCOL (so you can run the same input parser), or some abstraction you have layered on top of it. If you are converting a file (or EFI output) to input you need to convert CR+LF or LF to CR (for bonus points you could handle LF+CR and CR too).
 
Andrew Fish
 
PS Luckily for us subversion automatically can convert between line ending forms based on the OS you check the file out to. And most, good, editors can deal with different line ending schemes. I have run into situations where the line ending schemes got mixed and the editor got confused. I use bbedit, and there is a command to normalize line endings. 
 
On Jun 15, 2011, at 2:42 PM, Carsey, Jaben wrote:


I think this might be a compound problem.
 
If you run set after the type command what is the output?  The time command requires input without any carriage returns?  In reality almost no command is expecting \r or \n characters on the command line.
 
If you have a file with the time and then a trailing carriage return then you will have issues. 
 
Andrew could be correct that the environment variable replacement should modify the \r and \n characters to not be there, but I dont know what they should do.  Replace with spaces?  Chop them off??
 
-Jaben
 
From: Andrew Fish [mailto:afish@apple.com] 
Sent: Wednesday, June 15, 2011 2:35 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file
 
From a UEFI point of view. 
LF U+000A - Linefeed. Moves cursor to the next line.
CR U+000D - Carriage Return. Moves cursor to left margin of the current screen. 
 
So a Print (L"\n"); will emit \r\n and Print (L"\r"); outputs \r. Print (L"\r\n"); outputs \r\n
 
      case '\r':
        Format += BytesPerFormatCharacter;
        FormatCharacter = ((*Format & 0xff) | (*(Format + 1) << 8)) & FormatMask;
        if (FormatCharacter == '\n') {
          //
          // Translate '\r\n' to '\r\n'
          //
          ArgumentString = "\r\n";
        } else {
          //
          // Translate '\r' to '\r'
          //
          ArgumentString = "\r";
          Format   -= BytesPerFormatCharacter;
        }
        break;
 
Thus the type command is not broken. It is most likely the \r\n is not being handled correctly by the redirection input code in the shell. 
 
Andrew Fish
 
 



 
On Jun 15, 2011, at 2:01 PM, joelw@indyelectronics.com wrote:



After removing appending \r\n from type.c, using a unicode file I get a garbage character appended to said variable.  Looks like residue from variable list enumeration.  No garbage character when using an ANSI file as the source.
 

-----Original Message-----
From: joelw@indyelectronics.com [mailto:joelw@indyelectronics.com]
Sent: Wednesday, June 15, 2011 04:30 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file

Yep it is.  Line 86 of type.c:
 Print(L"\r\n", Buffer);
 

-----Original Message-----
From: Carsey, Jaben [mailto:jaben.carsey@intel.com]
Sent: Wednesday, June 15, 2011 04:16 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev_______________________________________________
edk2-devel mailing list
edk2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/edk2-devel
It sounds like the type command is appending a \r\n to the end of the line and then the time command does not like that.
 
-Jaben
 
From: joelw@indyelectronics.com [mailto:joelw@indyelectronics.com] 
Sent: Wednesday, June 15, 2011 1:14 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file
 
In my case I have a 8 byte file named time.txt, which contains "06:32:14".  I run
  type time.txt >v foo
  time %foo%
 
Output is:
  Error. The argument '06:32:14
  ' is incorrect.
 
If I do this:
  06:32:14 >v foo
  time %foo%
 
All is good with the world.
 
 
Are there any PCD values that might affect this behavior?
 
 
Regarding pre-fetch abort issue - if syntax of updating enviromental variable in split command line is not intended to be supported, then it should fail gracefully.
 

-----Original Message-----
From: Andrew Fish [mailto:afish@apple.com]
Sent: Wednesday, June 15, 2011 03:23 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file

Joel.
 
A pre-fetch abort would imply your code is running off into the weeds. 
 
I just tried:
type time.txt >v foo
time %foo%
 
where:
type time.txt
10:11:12
 
in the InOsEmuPkg X64 emulator and it works. 
 
I'm not sure you syntax is legal, but I can reproduce the crash in the InOsEmuPkg emulator. 
type time.txt >v foo|time %foo%
 
It looks like the FileHandle passed into FileHandleWrite is not a valid EFI_FILE_HANDLE. So the call to FileHandle->Write() is causing your pre-fetch abort. 
 
(gdb) p /x *FileHandle
$7 = {
  Revision = 0xafafafafafafafaf, 
  Open = 0xafafafafafafafaf, 
  Close = 0xafafafafafafafaf, 
  Delete = 0xafafafafafafafaf, 
  Read = 0xafafafafafafafaf, 
  Write = 0xafafafafafafafaf, 
  GetPosition = 0xafafafafafafafaf, 
  SetPosition = 0xafafafafafafafaf, 
  GetInfo = 0xafafafafafafafaf, 
  SetInfo = 0xafafafafafafafaf, 
  Flush = 0xafafafafafafafaf
}
 
 
#0  0x000000010aa6fae7 in FileHandleWrite (FileHandle=0x10ac5b318, BufferSize=0x106f56380, Buffer=0x10ac5101c) at /Users/fish/work/edk2/ShellPkg/Library/UefiFileHandleLib/UefiFileHandleLib.c:225
#1  0x000000010aa74ae2 in InternalPrintTo (String=0x10ac5101c) at /Users/fish/work/edk2/ShellPkg/Library/UefiShellLib/UefiShellLib.c:2548
#2  0x000000010aa75076 in InternalShellPrintWorker (Col=0xffffffff, Row=0xffffffff, Format=0x10ac5d898, Marker=0x106f56510) at /Users/fish/work/edk2/ShellPkg/Library/UefiShellLib/UefiShellLib.c:2654
#3  0x000000010aa75452 in ShellPrintHiiEx (Col=0xffffffff, Row=0xffffffff, Language=0x0, HiiFormatStringId=0x2, HiiFormatHandle=0x10afae118) at /Users/fish/work/edk2/ShellPkg/Library/UefiShellLib/UefiShellLib.c:2811
#4  0x000000010aaf59a3 in DoShellPrompt () at /Users/fish/work/edk2/ShellPkg/Application/Shell/Shell.c:943
#5  0x000000010aaf32ea in UefiMain (ImageHandle=0x10afb4198, SystemTable=0x106ef3f18) at /Users/fish/work/edk2/ShellPkg/Application/Shell/Shell.c:406
#6  0x000000010ab0a489 in ProcessModuleEntryPointList (ImageHandle=0x10afb4198, SystemTable=0x106ef3f18) at /Users/fish/work/edk2/Build/EmuUnixX64/DEBUG_XCLANG/X64/ShellPkg/Application/Shell/Shell/DEBUG/AutoGen.c:553
#7  0x000000010aa80450 in _ModuleEntryPoint (ImageHandle=0x10afb4198, SystemTable=0x106ef3f18) at /Users/fish/work/edk2/MdePkg/Library/UefiApplicationEntryPoint/ApplicationEntryPoint.c:65
#8  0x0000000106f6b4c7 in CoreStartImage (ImageHandle=0x10afb4198, ExitDataSize=0x106f56c20, ExitData=0x106f56c18) at /Users/fish/work/edk2/MdeModulePkg/Core/Dxe/Image/Image.c:1482
#9  0x0000000105f6ac81 in BdsLibBootViaBootOption (Option=0x10afb4398, DevicePath=0x10afb4418, ExitDataSize=0x106f56c20, ExitData=0x106f56c18) at /Users/fish/work/edk2/IntelFrameworkModulePkg/Library/GenericBdsLib/BdsBoot.c:798
#10 0x0000000105f8a689 in BdsBootDeviceSelect () at /Users/fish/work/edk2/IntelFrameworkModulePkg/Universal/BdsDxe/BdsEntry.c:221
#11 0x0000000105f8a264 in BdsEntry (This=0x105f97e60) at /Users/fish/work/edk2/IntelFrameworkModulePkg/Universal/BdsDxe/BdsEntry.c:363
#12 0x0000000106f81ee3 in DxeMain (HobStart=0x106eef018) at /Users/fish/work/edk2/MdeModulePkg/Core/Dxe/DxeMain/DxeMain.c:458
#13 0x0000000106f82361 in ProcessModuleEntryPointList (HobStart=0x103020000) at /Users/fish/work/edk2/Build/EmuUnixX64/DEBUG_XCLANG/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/AutoGen.c:318
#14 0x0000000106f5aea0 in _ModuleEntryPoint (HobStart=0x103020000) at /Users/fish/work/edk2/MdePkg/Library/DxeCoreEntryPoint/DxeCoreEntryPoint.c:54
#15 0x0000000106fcc8d4 in SwitchStack (EntryPoint=0x106fcf468 <GetNextHob+24>, Context1=0x10301ca78, Context2=0x106ff3340, NewStack=0x106fd5bd8) at /Users/fish/work/edk2/UnixPkg/Library/UnixBaseLib/SwitchStack.c:55
 
(gdb) l 221
216   )
217 {
218   //
219   // ASSERT if FileHandle is NULL
220   //
221   ASSERT (FileHandle != NULL);
222   //
223   // Perform the write based on EFI_FILE_PROTOCOL
224   //
225   return (FileHandle->Write(FileHandle, BufferSize, Buffer));
 
 
Andrew Fish
 
 
 
 
On Jun 15, 2011, at 11:36 AM, joelw@indyelectronics.com wrote:

 

Also hitting a pre-fetch abort with this:
 
type time.txt >v foo|time %foo%
 
 

-----Original Message-----
From: joelw@indyelectronics.com [mailto:joelw@indyelectronics.com]
Sent: Wednesday, June 15, 2011 02:23 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file

Hi Jaben,
 
Thanks for the clarification.  When I execute sequence, I see:
 
  Error. The argument '01:34:29
  ' is incorrect.
 
Running "set foo" I get "foo - 01:34:29", without a trailing /r or /n.
 
I created file with edit, and it does not have a training /r or /n.  Then running edit on the same file again, I can no longer save and get "Disk Error. Try Again".  A reboot clears up the disk error.  This is not related to block IO driver as system will boot from same block IO driver without issue.  The later would appear to be issue with "edit".
 
-joel 
 
-----Original Message-----
From: Carsey, Jaben [mailto:jaben.carsey@intel.com]
Sent: Wednesday, June 15, 2011 12:00 PM
To: edk2-devel@lists.sourceforge.net
Subject: Re: [edk2] Shell Piping - set time via text file

I think that there is some confusion between the command line for a shell command or shell application and the StdIn. The time command is passed the time to change to on its command line, it does not read this from StdIn. If you think of time as a standard C program, the argv[1] needs to be the time value to use. To change the command line passed into the time program, you would need to do something more like this: I am going to answer the rest from a UDK Shell 2.0 perspective. I am not certain which shell you are using. type time.txt >v foo time %foo% this would tell the shell to have the type command output sent to an environment variable, and then tell the shell to change the command line of the time command to use the environment variable foo. -Jaben -----Original Message----- From: Joel Winarske [mailto:joelw@indyelectronics.com] Sent: Tuesday, June 14, 2011 11:58 PM To: edk2-devel@lists.sourceforge.net Subject: [edk2] Shell Piping - set time via text file I'm trying to set time using a text file in Shell. I have a text file with time value, and I issue: type time.txt|time Time is not set to value in text file. Text file contains time in same format that one would pass to time from Shell prompt. Is this a bug, or by design? -joel ------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ edk2-devel mailing list edk2-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/edk2-devel ------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ edk2-devel mailing list edk2-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/edk2-devel
------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev_______________________________________________
edk2-devel mailing list
edk2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/edk2-devel
 
 
------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev_______________________________________________
edk2-devel mailing list
edk2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/edk2-devel
 
------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev_______________________________________________
edk2-devel mailing list
edk2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/edk2-devel