Converting between Eclipse and Windows Newlines
The differences between Unix and Linux with regard to line endings in editors can often cause issues. Many people run their IDEs on Microsoft Windows then transfer their code to Unix. This is generally fine for code which is usually compiled, but for scripts that are run straight on the Unix box this can cause issues.
If you attempt to run a script on Unix that has windows line endings, you will errors messages such as “Not Found” when you try to run these scripts, even though on the face of it these scripts look normal.
I use Eclipse on Windows to create my Unix scripts and recently came across this problem at work. I went to run a new script I had deployed and got the “Not Found” message even though everything looked normal on the face of it. Checking the line endings it turned out that my script had Windows line endings and not Unix ones.
What Are The Correct New Lines For Unix
When you edit text files in Windows it will use windows line endings. Windows ends lines with a carriage return “\r” and a line feed “\n”. Unix only uses a line feed, and this is what it expects to find in its scripts or it will complain and give unexpected results.
The solution in Eclipse consisted of two things. The first was to set Eclipse to use Unix line endings. Although Unix doesn’t like Windows line endings, Windows will work quite happily with files that have Unix line endings.
Under Preferences, General, Workspace there is a setting for “new text file line delimiter”. Setting this to ‘Unix’ will ensure the right line endings are used for any new text files that are created. I had recently upgraded by Eclipse from Keplar to Luna, so I suspect this setting somehow got reverted during the upgrade process, asI didn’t have the line ending issue before, and we have a lot of Unix scripts in our application.
The above solves the issue for any new files you create. But what about the existing files you have created. Well we needed a way to track those down so that we could ensure we fixed any additional scripts that needed to be changed to have Unix line endings. The way to do this is in the search tool. By clicking on the regular expression option, this allows you to search for control characters such as the carriage return character.
Using this will give you a list of all your Unix scripts that have Windows line endings. Ok so you have found them, but how do you change them to Unix line endings. We theres also a way to that in Eclipse too. File, Convert Line Delimiters To gives you the option to convert between Windows and Unix as required.
Hopefully the above will help you if you get into the same situation as I did with Unix scripts created on Windows failing.
You can also convert your file at a project or folder level too. Select your project, and then do File, Convert Line Delimiters To, and it will convert all the files in your project.
There are a few other tools that are handy when converting between windows and unix line formats too.
Windows
Notepad++ also have the feature to convert between windows and unix line endings that will convert crlf to lf and lf to crlf, which is handy if you are updating scripts outside of your IDE. If you click edit, EOL conversions… you will see the option to convert between windows and unix line ending formats.
Unix
Dos2Unix will convert files from windows/mac format line endings to unix line endings. This is handy especially if you are bring over files edited on other systems, especially from vendors where you arent sure where the files have been edited or if they are in the correct format.
Automatically Checking For Windows Line Endings in Unix Scripts
If we wanted to create something to validate that are scripts are in the correct format before we deploy or commit our code, the simplest way to do it is to scan for the carriage return control character in our unix scripts. So if we use the code similar to below
String stringFile = FileUtils.readFileToString(file);
int tokens = new StringTokenizer(stringFile, "\r").countTokens();
if (tokens > 0) {So what we are doing here is checking if the file has carriage returns in it, because if it has it means its in windows line ending format. You can create a junit test around this that you can run with your other junit tests so it will fail if you pass this a unix script file to validate.
I would suggest grabbing a list of all your unix script files and calling a method that does the above in a loop, and failing if it finds any in windows format.
Git autocrlf and Eclipse
If you use Git with Eclipse, then you may suffer from issues with automatic end of line conversion between eclipse and git repositories. As specified on NetMikey’s blog, you should always set and check the global value to ensure your line endings dont switch from unix to windows line endings.
core.autocrlf option should be set globally to true ALWAYS!!!
To check the current setting:
git config --global core.autocrlf
which you want to come back as true.
To set it:
git config --global core.autocrlf true