unity3d - Syntax error in one of two almost-identical batch scripts: ")" cannot be processed syntactically here

Question

I am trying to set up a Jenkins server for automatic Unity builds.

Therefore I have written two (in my eyes) basically identical batch scripts.

Both scripts are run as build steps by Jenkins via an Execute Windows batch command step using

Command: E:unityImport.bat

and after that a second Execute Windows batch command step using

Command: E:unityBuild.bat

They both have the same beginning as I need to gather some file paths and in particular the project's unity version. So in both scripts I use exactly the same way of parsing and string splitting the project version. The only thing that is different between them is that the first one starts Unity and imports a dedicated unitypackage (which hold the method to execute in the next step) into the project while the second one again starts Unity to perform the actual build (unfortunately it didn't work in one single go ... Unity seems to try to execute the method before the unitypackage is imported).

However the second script always fails with a syntax error

")" cannot be processed syntactically here.

---

What I try to achieve is

1. Read out the content of the file %WORKSPACE%ProjectSettingsProjectVersion.txt

   SET /p TEST=<%WORKSPACE%ProjectSettingsProjectVersion.txt
   

   The content of %TEST% usually looks like e.g.

   m_EditorVersion: 2019.3.4f1
   

   and ECHO. ProjectVersion.txt = %TEST% looks like

   ProjectVersion.txt = m_EditorVersion: 2019.3.4f1
   

2. string split in order to only take the last part containing the version number

   for %%x in (%TEST::= %) do (
       SET "VALUE=%%x"
       SET "UNITY_VERSION=!VALUE:~0,-2!" 
   )
   

   so %UNITY_VERSION% usually contains e.g. 2019.3.4. I don't split more away because there are also Unity version with two digits like e.g. 2018.4.18

3. string split on the . in order to only get the major release number

   for /f "tokens=1,2 delims=." %%a in ("%UNITY_VERSION%") do (
       SET "A=%%a"
       SET "B=%%b"
   )
   SET "UNITY_VERSION=%A%.%B%"
   

   which results in %UNITY_VERSION% being e.g. 2019.3

4. Finally search in all installed Unity versions if the required version is present

   set "UNITY_FOLDER="
   for /f "delims=" %%a in ('dir /b E:Unity\%UNITY_VERSION%*') do (
       set "UNITY_FOLDER=%%a"
   )
   

   after this we either found a valid Unity installation folder for the given version or not.

---

So here are the scripts.

Import (This works as expected)

@ECHO OFF
CLS
ECHO.

cd %WORKSPACE%

IF NOT EXIST %WORKSPACE%ProjectSettingsProjectVersion.txt (
    EXIT 1
)

SETLOCAL ENABLEDELAYEDEXPANSION

SET /p TEST=<%WORKSPACE%ProjectSettingsProjectVersion.txt
ECHO. ProjectVersion.txt = %TEST%

for %%x in (%TEST::= %) do (
    SET "VALUE=%%x"
    SET "UNITY_VERSION=!VALUE:~0,-2!" 
)

for /f "tokens=1,2 delims=." %%a in ("%UNITY_VERSION%") do (
    SET "A=%%a"
    SET "B=%%b"
)

SET "UNITY_VERSION=%A%.%B%"
ECHO. Project Unity Version = %UNITY_VERSION%

set "UNITY_FOLDER="
for /f "delims=" %%a in ('dir /b E:Unity\%UNITY_VERSION%*') do (
    set "UNITY_FOLDER=%%a"
)

IF "%UNITY_FOLDER%"=="" (
    EXIT 1
)

ECHO. Using Unity Version %UNITY_FOLDER%

ECHO. Running:
ECHO. E:Unity\%UNITY_FOLDER%EditorUnity.exe -quit -batchmode -projectPath %WORKSPACE% -logFile - -importPackage E:UnityBuildPackageAutoBuilder.unitypackage

E:Unity\%UNITY_FOLDER%EditorUnity.exe -quit -batchmode -projectPath %WORKSPACE% -logFile - -importPackage E:UnityBuildPackageAutoBuilder.unitypackage

IF NOT %errorlevel% equ 0 (
    EXIT 1
) 

EXIT 0

Build (This fails with a syntax error I will mark with REM HERE IT BREAKS! ... which is not present in the actual script)

@ECHO OFF
CLS
ECHO.

cd %WORKSPACE%

IF NOT EXIST %WORKSPACE%ProjectSettingsProjectVersion.txt (
    EXIT 1
)

SETLOCAL ENABLEDELAYEDEXPANSION

SET /p TEST=<%WORKSPACE%ProjectSettingsProjectVersion.txt
ECHO. ProjectVersion.txt = %TEST%

REM HERE IT BREAKS! The before echo is the last I see before getting the syntax error

for %%x in (%TEST::= %) do (
    SET "VALUE=%%x"
    SET "UNITY_VERSION=!VALUE:~0,-2!" 
)

for /f "tokens=1,2 delims=." %%a in ("%UNITY_VERSION%") do (
    SET "A=%%a"
    SET "B=%%b"
)

SET "UNITY_VERSION=%A%.%B%"
ECHO. Project Unity Version = %UNITY_VERSION%

set "UNITY_FOLDER="
for /f "delims=" %%a in ('dir /b E:Unity\%UNITY_VERSION%*') do (
    set "UNITY_FOLDER=%%a"
)

IF "%UNITY_FOLDER%"=="" (
    EXIT 1
)

ECHO. Using Unity Version %UNITY_FOLDER%

...

I don't think the rest matters since as said What I see in the console it already breaks after e.g.

ProjectVersion.txt = 2019.3.4f1

")" cannot be processed syntactically here.

---

Does anyone see the mistake or is there maybe something with Jenkins that makes the second script fail with a syntax error even though as far as I see they are basically identical?

Answer 1

There are multiple small issues with the code which I explain one after the other below my suggestion for the batch file.

The task to get the UNITY_FOLDER according to UNITY_VERSION as defined in file ProjectVersion.txt can be done more efficient by using the following code:

@echo off
setlocal EnableExtensions DisableDelayedExpansion

if not defined WORKSPACE (
    echo ERROR: Environment variable WORKSPACE is not defined.
    exit /B 1
)

if not exist "%WORKSPACE%ProjectSettingsProjectVersion.txt" (
    echo ERROR: File "%WORKSPACE%ProjectSettingsProjectVersion.txt" does not exist.
    exit /B 1
)

set "UNITY_FOLDER="
set "UNITY_VERSION="
for /F "usebackq tokens=2-4 delims=. " %%I in ("%WORKSPACE%ProjectSettingsProjectVersion.txt") do (
    if not "%%~K" == "" (
        for /F "delims=abcdef" %%L in ("%%~K") do (
            set "UNITY_VERSION=%%~I.%%~J.%%~L"
            for /D %%M in ("E:Unity\%%~I.%%~J*") do set "UNITY_FOLDER=%%M"
        )
    )
)

if not defined UNITY_VERSION (
    echo ERROR: Failed to determine unity version from "%WORKSPACE%ProjectSettingsProjectVersion.txt".
    exit /B 1
)
if not defined UNITY_FOLDER (
    echo ERROR: Failed to find a folder in "E:Unity" for unity version %UNITY_VERSION%.
    exit /B 1
)

echo Found for unity version %UNITY_VERSION% the folder "%UNITY_FOLDER%".

cd /D "%WORKSPACE%" 2>nul
if errorlevel 1 (
    echo ERROR: Failed to set "%WORKSPACE%" as current folder.
    exit /B
)
rem Other commands to execute.

endlocal

This batch file first sets up the execution environment required for this batch file using command SETLOCAL.

The existence of the environment variable WORKSPACE is verified next by the batch file. This environment variable should be defined by Jenkins outside this batch file. An error message is output on missing definition of this important environment variable.

Then the existence of the text file is checked with printing an error message if not existing and exiting batch file with exit code 1.

The two environment variables UNITY_FOLDER and UNITY_VERSION are deleted if defined by chance outside the batch file.

Next the text file is processed which should contain only one non-empty line with the data of interest. Otherwise it would be necessary to change the code to evaluate also the first substring if being equal m_EditorVersion: before execution of the other commands.

FOR with option /F interprets a set enclosed in " by default as string to process. But in this case the string in " should be interpreted as full qualified file name of the file of which contents should be processed line by line by FOR. For that reason the option usebackq is used to get the wanted file contents processing behavior.

FOR ignores always empty lines on processing the contents of a file. So it would not matter if the text file contains at top one or more empty lines.

FOR splits up a line by default into substrings using normal space and horizontal tab character as string delimiters. If the first space/tab delimited string starts with a semicolon being the default end of line character after removing all leading spaces/tabs, the line would be also ignored by FOR like an empty line. Finally just the first space/tab delimited string would be assigned to the specified loop variable I.

This default line processing behavior is not wanted here because of getting just m_EditorVersion: assigned to the specified loop variable I is not enough. For that reason the option delims=.? is used to get the line split up on dots and spaces. The option tokens=2-4 informs FOR that the second space/dot delimited substring 2019 should be assigned to loop variable I, the third space/dot delimited substring 3 to next loop variable J which is the next character in the ASCII table and the fourth space/dot delimited substring 4f1 to next but one loop variable K.

It is important here to specify delims=.? at end of the options argument string with the space character as last character because of the space character is otherwise interpreted as an options separating character to ignore like the space between usebackq and tokens=2-4 and the space between tokens=2-4 and delims=.?. In fact it would be also possible to write the options without spaces like "usebackqtokens=2-4delims=. ", but that makes the argument string with the options difficult to read.

The default end of line definition eol=; can be kept here because of the line with the unity version in ProjectVersion.txt does not have a semicolon after 0 or more spaces/dots and is never ignored for that reason.

FOR runs the commands in the command block on having found in line at least the second space/dot delimited string assigned to loop variable I, i.e. a non-empty string is assigned to specified loop variable I. But the commands should be executed only if all three parts of the unity version were determined by FOR and assigned to the loop variables I, J and K. Therefore a simple string comparison is made to verify that loop variable value reference %%~K does not expand to an empty string as that would mean not having read enough parts of unity version from?the file.

I don't know what f1 at end of the editor version means. So one more FOR with option /F is used to split the string 4f1 (no usebackq on string enclosed in ") into substrings using the characters abcdef (lower case hexadecimal characters) as string delimiters and get assigned to specified loop variable L just the first substring. That should never fail and so environment variable UNITY_VERSION is defined with 2019.3.4.

The third FOR is executed inside the second FOR although it could be also outside because of not referencing loop variable L. So the following code could be also used here with the same result.

for /F "usebackq tokens=2-4 delims=. " %%I in ("%WORKSPACE%ProjectSettingsProjectVersion.txt") do (
    if not "%%~K" == "" (
        for /F "delims=abcdef" %%L in ("%%~K") do set "UNITY_VERSION=%%~I.%%~J.%%~L"
        for /D %%M in ("E:Unity\%%~I.%%~J*") do set "UNITY_FOLDER=%%M"
    )
)

FOR with option /D and a set containing * (or ?) results in searching in specified directory E:Unity for a non-hidden directory of which name starts with 2019.3. Each non-hidden directory in E:Unity matching the wildcard pattern 2019.3* is assigned one after the other with full qualified name (drive + path + name) first to loop variable M and next to environment variable UNITY_FOLDER. FOR never encloses itself a file/folder string in " which is the reason why %%M can be used here and %%~M is not necessary. The folder name assigned to loop variable M is never enclosed in " in this case. So the environment variable UNITY_FOLDER contains the last folder matching the wildcard pattern returned by the file system with full path. This means on multiple folder names matching the wildcard pattern 2019.3* that the file system determines which folder name is assigned last to UNITY_FOLDER. NTFS stores directory entries in its master file table sorted in a local specific alphabetic order while FAT, FAT32 and exFAT store directory entries unsorted in their file allocation tables.

Note: If the third number of editor version is not really needed as it looks like according to the code in question, it would be also possible to use:

for /F "usebackq tokens=2-4 delims=. " %%I in ("%WORKSPACE%ProjectSettingsProjectVersion.txt") do (
    if not "%%~J" == "" (
        set "UNITY_VERSION=%%~I.%%~J"
        for /D %%K in ("E:Unity\%%~I.%%~J*") do set "UNITY_FOLDER=%%K"
    )
)

Two additional checks are made if the code could successfully determine unity version and find a matching unity folder.

The echo command line at bottom of the batch file is just for verification of the result on running this batch file with WORKSPACE defined outside the batch file in command prompt window and everything worked as expected.

There is no need to make the workspace directory the current directory up to end of the batch file, but?I added the code to do that with the verification if changing the current directory to workspace directory was done really successfully.

---

Issue 1: File/folder argument strings not enclosed in quotes

The help output on running in a command prompt cmd /? explains with last paragraph on last page that a file/folder argument string containing a space or one of these characters &()[]{}^=;!'+,`~ requires surrounding straight double quotes. So it is advisable to always enclose file/folder names without or with path in ", especially on one or more parts being dynamically defined by an environment variable or being read from file system.

So not good are the following command lines:

cd %WORKSPACE%
IF NOT EXIST %WORKSPACE%ProjectSettingsProjectVersion.txt
SET /p TEST=<%WORKSPACE%ProjectSettingsProjectVersion.txt

Better would be using:

cd "%WORKSPACE%"
IF NOT EXIST "%WORKSPACE%ProjectSettingsProjectVersion.txt"
SET /p TEST=<"%WORKSPACE%ProjectSettingsProjectVersion.txt"

It can be read in short help output on running cd /? that the command CD does not interpret a space character as argument separator like it is the case for most other internal commands of Windows command processor cmd.exe or executables in directory %SystemRoot%System32 which are installed by default and also belong to the Windows commands according to Microsoft. But changing the current directory fails on omitting " if the directory path contains by chance an ampersand because of & outside a double quoted argument string is interpreted already by cmd.exe as AND operator before execution of CD as described for example in my answer on <a href="https://stackoverflow.c