Preface

Previously the EBS JAR signing has been performed using adadmin. But the rules have changed and starting June 1st 2023 all new code-signing certificates utilize an HSM (Hardware Security Module) or Token. And the signing is now performed using a utility provided by the CA. In this article I will provide instructions to sign the EBS JAR files using the new procedure. I used that procedure to sign jars in our EBS Demo instance which we use to show Blitz Report capabilities.

Identifying the list of available certificate authorities

The prices for the new code-signing certificates are substantially higher than for the old ones. So I had to perform a market research to find the best offer. The major CAs provide the code signing certificates with the new technology. It is better to chose the one included in the JRE cacerts file on the clients’ machines as noted on this post.

To find a list of CAs in the JRE on the Windows desktop, run the following command in cmd. Adjust the JRE location. The default password is changeit.

"C:\Program Files (x86)\Java\jre-1.8\bin\keytool" -list -keystore "C:\Program Files (x86)\Java\jre-1.8\lib\security\cacerts" -v |findstr Issuer:

But the output of the above command is difficult to analyze as it is very large. Also not all the CAs from the output still issue the code signing certificates, e.g. SwissSign. Some of the CAs redirect to other CAs’ websites. E.g. QuoVadis redirects to the Digicert website.

I found the following shorter list of CAs from Microsoft, which may not be as comprehensive, but easier to check.

Here is the price comparison I prepared. All the information was obtained from the CAs’ websites.

Certificate authority Price for 3 years
Digicert $1806
Certum €379.00
Entrust $893.18
Globalsign $950 ($315/year)
Identrust $570 (Only EV certificates available)
Sectigo $880
SSL.com $328 plus $540 esigner subscription or $249 for a USB key

Of course the price is a big factor. The other important factor was the electronic signer availability as I did not want to deal with the hardware tokens shipped via mail. Considering the above I chose the Certum CA.

Order a code signing certificate.

This step should be similar for all the CAs. I ordered the code signing certificate for 3 years at the Certum’s website. The next step is to activate the certificate and provide the information about the subscriber: name, surname, email address, organization name, etc. After some time Certum sends a link to perform the identity verification. It is totally remote and automated which is cool. It is required to upload the documents like passport and then take a picture of your face using the webcam. The verification took a couple of days and then I got the confirmation email.

Activating the certificate

The next step was to activate the issued certificate. Certum provides the following instructions to do that:

https://support.certum.eu/en/how-to-activate-code-signing-simplysign/
https://www.files.certum.eu/documents/manual_en/Code-Signing-SimplySign-Instructions-for-activation-and-installation-v1.1.pdf

Signing the JARs in Windows

Certum provides the signing software for Windows, Linux and OS X platforms. Depending on your requirements you may choose to sign the EBS JARs in place on the Linux apps server or copy the files over to a separate Windows/OS X server and then copy them back.

Download the required software.

It is required to download and install the SimplySign mobile application from the Google Play or App Store. The mobile application generates constantly changing codes which are required to login to the desktop application.

Download and install proCertum SmartSign. It includes SimplySign Desktop which is not available separately.

After the SimplySign Desktop application is started it will continue to run in background, you may find its icon in the Windows system tray. The application itself does not sign files, but acts as the certificates storage.  It is required to configure the jarsigner to sign the JARs. It is described in the next section.

Configuring jarsigner

Certum provides the following instructions for configuring jarsigner:

https://support.certum.eu/en/signing-the-code-using-tools-like-signtol-and-jarsigner-instruction/
https://www.files.certum.eu/documents/manual_en/Code-Signing-signing-the-code-using-tools-like-Singtool-and-Jarsigner_v2.3.pdf

Install JDK

Download and install the latest JDK for Windows. Jarsigner is located under the JDK installation directory. E.g. “C:\Program Files\Java\jdk-21\bin\jarsigner.exe”.

Add the path to jarsigner to the Path variable

Add jarsigner to path

Create a configuration file provider.cfg

The file should contain:

name=SimplySignPKCS.dll
library=C:\Windows\System32\SimplySignPKCS.dll
slot=-1

Create certificate path file bundle.pem


Use the Certum instructions to create the bundle.pem

It is required to combine 3 certificates in bundle.pem:

1. “Above”: User Certificate
2. “Below”: intermediate certificate for user certificate
3. “Bottom”: cross certificate

So the resulting file looks similar to the following short representation:

-----BEGIN CERTIFICATE-----
MIIHBTCCBO2gAwIBAgIQMcbBECbTIn+xk+i+JKLv/DANBgkqhkiG9w0BAQsFADBW
u7bKugOItNs4RTrfjG7EN11y1lvVF5vEJQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIGuTCCBKGgAwIBAgIRAJmjgAomVTtlq9xuhKaz6jkwDQYJKoZIhvcNAQEMBQAw
5hZ59IGynk08mHhBFM/0MLeBzlAQq1utNjQprztZ5vv/NJy8ua9AGbwkMWkO
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFyTCCBLGgAwIBAgIQG7WPJSrfIwBJKMmuPX7tJzANBgkqhkiG9w0BAQwFADB+
TtBKF0NMxfGnbNIeWGwUUX6KVKH27595el2BmhaQD+G78UoA+fndvu2q7M4K
-----END CERTIFICATE-----

Place the files under a separate directory, e.g. C:\ebsjarsigning.

Run jarsigner

Run jarsigner to sign a test JAR file. Execute the following command in Windows command prompt to get the certificate alias name:

keytool -list -v -keystore NONE -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg "C:\ebsjarsigning\provider.cfg" -storepass ""
keytool get alias

Use the alias name to construct the correct jarsigner command. The storepass can be left empty:

jarsigner -keystore NONE -certchain "C:\ebsjarsigning\bundle.pem" -tsa "http://time.certum.pl" -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg "C:\ebsjarsigning\provider.cfg" -storepass "" "C:\users\Admin\Downloads\fndaol.jar" "31C6C11026D3227FB193E8BE24A2EFFC"
 
jar signed.
 
The signer certificate will expire on 2026-09-03.
The timestamp will expire on 2029-09-17.

Sign the EBS jars in batch

The following MOS document contains the instructions for the new jar signing procedure:

Signing EBS Jar Files With HSM (Hardware Security Module) (Doc ID 2806640.1)

Run adadmin

Generate a list of jar files to sign by running ADADMIN, and select the following from the AD Administration Main Menu:

Choose Generate Applications Files menu
From this menu choose Generate product jar files
Enter yes when prompted with: Do you wish to force regeneration of all jar files? [No] ? yes

Compress the JARs to be signed.

All the files listed in jarlist.txt located in the following directories:

EBS 11i and r12.1

$APPL_TOP/admin/$TWO_TASK/log

EBS 12.2

$NE_BASE/EBSapps/log/adadmin/log

You may run the following Linux script to create a backup of the JAR files to be signed, remove the adadmin signature and compress the files into a single archive. This archive may be copied to a server where the signing process takes place. When prompted for the operation, enter ‘copy‘.

Sign the files

Unzip the archive to a staging directory, where the signing occurs, e.g.

C:\Users\admin\Downloads\jars_to_sign

Run the following command to find all the jar files under the top directory recursively and sign them:

for /R "C:\Users\admin\Downloads\jars_to_sign" %f in ("*.jar") do jarsigner -keystore NONE -certchain "C:\ebsjarsigning\bundle.pem" -tsa "http://time.certum.pl" -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg "C:\ebsjarsigning\provider.cfg" -storepass "" "%f" "31C6C11026D3227FB193E8BE24A2EFFC"

After the files are signed, compress them into a new archive jars_signed.zip.

Restore the signed files on the run fs

Transfer the archive jars_signed.zip back to the EBS application server.
Uncompress the archive and copy the files back to their original locations:

unzip jars_signed.zip
cp -Rv /tmp/jars_signed/u01 /
 
‘u01/install/APPS/fs1/EBSapps/comn/java/classes/customall.jar’ -> ‘/u01/install/APPS/fs1/EBSapps/comn/java/classes/customall.jar’
‘u01/install/APPS/fs1/EBSapps/comn/java/classes/oracle/apps/ad/jar/adxlib.jar’ -> ‘/u01/install/APPS/fs1/EBSapps/comn/java/classes/oracle/apps/ad/jar/adxlib.jar’
‘u01/install/APPS/fs1/EBSapps/comn/java/classes/oracle/apps/ak/jar/akobjnav.jar’ -> ‘/u01/install/APPS/fs1/EBSapps/comn/java/classes/oracle/apps/ak/jar/akobjnav.jar’
…..

Restore the signed files on the patch fs for EBS 12.2

To use the same signed files for the patch fs rename the fs1 to fs2 or vice versa.

cd /tmp/jars_signed/u01/install/APPS/
mv fs1 fs2
cd /tmp
cp -Rv /tmp/jars_signed/u01 /

Signing the JARs on Linux

I used the following post by Johannes Michler as a source of information for this section.

Signing the JARs on Windows has its advantages: you do not have to install additional software on the EBS apps server. It may be restricted by your organization’s security policies. But it makes the signing process complicated.

The alternative is to configure the signing software on the Linux EBS apps server. I tested the steps on Oracle Linux 7.9.

Install the requred software

The Certum signing software requires a desktop. You have a couple of alternatives.

Configure the VNC server

To use the VNC for connection you need to install GUI. For example to install GUI in Oracle Linux 7:

yum groupinstall "server with gui"

Then you need to configure VNC on Linux like described on the following link.
The advantage of this approach is that you will have a convenient and beautiful desktop environment.
But it also requires a lot of disk space, RAM. It also requires a firewall exclusion or SSH port forwarding to connect.

Configure X11 forwarding

The alternative to the VNC is to use X11 forwarding which displays a Linux server application on your desktop.

Install required libraries and stalonetray.

Stalonetray stand-alone system tray (notification area) for X Window System/X11.

yum install libxslt.x86_64 pulseaudio-libs-glib2.x86_64 libwebp.x86_64
wget https://www.rpmfind.net/linux/epel/8/Everything/x86_64/Packages/s/stalonetray-0.8.3-15.el8.x86_64.rpm
yum install stalonetray-0.8.3-15.el8.x86_64.rpm
Install Mobaxterm

Download and install Mobaxterm on your desktop. It supports X11 forwarding to display the Linux server applications GUI on your desktop. To test that this feature works connect to your apps server and run a test application with GUI. Usually ‘xclock’ is used for that purpose.

test x11 forwarding mobaxterm

Install SimplySignDesktop

Download and install the latest version of SimplySignDesktop for Linux. The binary should be executed as a non privileged user but later asks for the root password.

su - applmgr
chmod u+x SimplySignDesktop-2.9.8-9.1.6.0-x86_64-prod-centos.bin
sh SimplySignDesktop-2.9.8-9.1.6.0-x86_64-prod-centos.bin

Start stalonetray and SimploySIgnDesktop in a MobaXterm session. I use the following script which stops the old processes and starts the new ones:

killall stalonetray
stalonetray &
killall SimplySignDesktop
/opt/SimplySignDesktop/SimplySignDesktop_start &

After the application starts it displays the small icon. On my Windows 11 it appeared in the left upper corner. Right click on it and choose ‘Connect with Cloud’.

Start SimplySignDesktop Certum Linux

On the next window provide your Certum login and the token from the mobile app. You may need to type them in a notepad on Windows and copy/paste into the login window as typing directly did not work for me.

Login SimplySignDesktop Certum Linux

Configure SimplySignDesktop

Create a configuration file

Create a configuration file, e.g. provider_simplysign.cfg with the following contents:

name=SimplySignDesktop/SimplySignPKCS
library=/opt/SimplySignDesktop/SimplySignPKCS_64-MS-1.0.20.so
slot=-1

Run keytool

Run a test keytool command to verify that SimplySignDesktop works:

/opt/proCertumSmartSign/jre/bin/keytool -list -keystore NONE -storetype PKCS11 -providerclass sun.security.pkcs11.SunPKCS11 -providerArg /home/applmgr/jarsigning/provider_simplysign.cfg -v -storepass ""
Run keytool Certum

Take a note of the Alias name. It will be required later on.

Create certificate path file bundle.pem

Create bundle.pem using as was described in the Windows section.

Test signing

To test the signing copy a JAR from $JAVA_TOP to a temp directory and run the following command. Adjust the command with the actual location of the files.

[applmgr@r122_Local tmp]$ cp $JAVA_TOP/oracle/apps/fnd/jar/fndall.jar /tmp
[applmgr@r122_Local tmp]$ /opt/proCertumSmartSign/jre/bin/jarsigner -keystore NONE -tsa "http://time.certum.pl" -certchain "/home/applmgr/jarsigning/bundle.pem" -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg "/home/applmgr/jarsigning/provider_simplysign.cfg" -storepass "12345" "/tmp/fndall.jar" 31C6C11026D3227FB193E8BE24A2EFFC
 
jar signed.
 
The signer certificate will expire on 2026-09-03.
The timestamp will expire on 2029-09-17.

Automating the signing on Linux

To automate the signing you may customize and use the following script.

Healthcheck

Verify the jar files signature

Run the following command on the EBS server:

[applmgr@demo APPS]$ jarsigner -verify -verbose /u01/install/APPS/fs1/EBSapps/comn/java/classes/oracle/apps/xtr/jar/xtrintrp.jar

It should return the certificate details

- Signed by "CN=Enginatics GmbH, O=Enginatics GmbH, OU=Software Engineering, L=Küsnacht, ST=Zürich, C=CH"
    Digest algorithm: SHA-256
    Signature algorithm: SHA256withRSA, 2048-bit key
- Signed by "[email protected], CN=Enginatics GmbH, O=Enginatics GmbH, OU=Software Engineering, L=Küsnacht, ST=Zürich, C=CH"
    Digest algorithm: SHA-256
    Signature algorithm: SHA384withRSA, 4096-bit key
  Timestamped by "CN=Certum Timestamp 2022, O=Asseco Data Systems S.A., C=PL" on Tue Sep 12 03:34:54 UTC 2023
    Timestamp digest algorithm: SHA256
    Timestamp signature algorithm: SHA384withRSA, 4096-bit key

EBS Services restart

It is not required to restart any service for EBS 11i.

For EBS 12.2 restart all the EBS oacore servers.

Otherwise the following exception may occur:

ExitException[ 3]com.sun.deploy.net.FailedDownloadException: Unable to load resource: https://demo.enginatics.com:4443/OA_JAVA/oracle/apps/fnd/jar/fndformsi18n.jar
	at sun.plugin2.applet.JNLP2Manager.downloadResources(Unknown Source)
	at sun.plugin2.applet.JNLP2Manager.prepareLaunchFile(Unknown Source)
	at sun.plugin2.applet.JNLP2Manager.loadJarFiles(Unknown Source)
	at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

Open any EBS form

Accept the following prompt which is going to display for the first form opening:

Java publisher warning
Java publisher warning

After confirming the prompts the form should successfully open.

Signed jar opened

Conclusion

It is not possible to provide the generic instructions for signing the EBS jars using HSM as each CA has its own procedure. But I hope this post gives you a high-level plan and helps to sign your EBS instances.

Appendix: JAR signing script.

This script may be customized according to your CA requirements. Replace the signing section which is currently uses the Certum signing software. Or if you use Certum you still need to adjust the paths to the configuration files.

/opt/proCertumSmartSign/jre/bin/jarsigner -keystore NONE -tsa "http://time.certum.pl" -certchain "/home/applmgr/jarsigning/bundle.pem" -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg "/home/applmgr/jarsigning/provider_simplysign.cfg" -storepass "12345" "${jar}" 31C6C11026D3227FB193E8BE24A2EFFC

The full script code:

printf '%s\n'
printf '%s\n' "The script support the following operations:"
printf '%s\n'
printf '%s\n' "sign - signs the JAR files right on this server using your custom command."
printf '%s\n' "copy - creates an archive with JAR files to be signed on a remote server."
printf '%s\n'
read -p "Enter the required operation: " operation
  case "$operation" in
    [sS][iI][gG][nN])
      sign_jars='Y'
      ;;
    [cC][oO][pP][yY])
      sign_jars='N'
      ;;
    *)
      printf '%s\n'
      printf '%s\n' "Invalid input. Aborting the script execution."
      printf '%s\n'
      exit 1
      ;;
  esac
#Combining the two possible sources of JARs that require signing
adjarlist="/tmp/jars_to_sign.txt"
:>$adjarlist
if [ -z "$NE_BASE" ]; then
  printf '%s\n'
  printf '%s\n' "NE_BASE is not set. Assuming the EBS version is 11i or R12.1"
  printf '%s\n'
  if [ -s "$APPL_TOP/admin/$TWO_TASK/log/jarlist.txt" ]; then
    cat "$APPL_TOP/admin/$TWO_TASK/log/jarlist.txt">>"$adjarlist"
  fi
  if [ -s "$APPL_TOP/admin/$TWO_TASK/log/jarlist.bak" ]; then
    cat "$APPL_TOP/admin/$TWO_TASK/log/jarlist.bak">>"$adjarlist"
  fi
else
  printf '%s\n'
  printf '%s\n' "NE_BASE is set. Assuming the EBS version is 12.2"
  printf '%s\n'
  if [ -s "$NE_BASE/EBSapps/log/adadmin/log/jarlist.txt" ]; then
    cat "$NE_BASE/EBSapps/log/adadmin/log/jarlist.txt">>"$adjarlist"
  fi
  if [ -s "$NE_BASE/EBSapps/log/adadmin/log/jarlist.bak" ]; then
    cat "$NE_BASE/EBSapps/log/adadmin/log/jarlist.bak">>"$adjarlist"
  fi
fi
if [ -s "$adjarlist" ]; then
  jars_to_sign=`cat $adjarlist | grep '\.jar'`
  printf '%s\n'
  printf '%s\n' "Backing up the JARs to the /tmp/backup_jars_`date +%F`.zip"
  printf '%s\n'
  rm -f /tmp/backup_jars_`date +%F`.zip
  echo "$jars_to_sign"| zip -@ /tmp/backup_jars_`date +%F`.zip
 
  if [ "$sign_jars" = 'Y' ]; then
    printf '%s\n'
    printf '%s\n' "Signing the JAR files."
    printf '%s\n'
    for jar in ${jars_to_sign}
    do
      printf '%s\n'
      printf '%s\n' "Removing EBS adadmin signature from: ${jar}"
      printf '%s\n'
      zip -d "$jar" 'META-INF/*.SF' 'META-INF/*.RSA'
      printf '%s\n'
      /opt/proCertumSmartSign/jre/bin/jarsigner -keystore NONE -tsa "http://time.certum.pl" -certchain "/home/applmgr/jarsigning/bundle.pem" -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg "/home/applmgr/jarsigning/provider_simplysign.cfg" -storepass "12345" "${jar}" 31C6C11026D3227FB193E8BE24A2EFFC
    done
    printf '%s\n'
    printf '%s\n' "Script finished at: `date`"
    printf '%s\n'
  else
    printf '%s\n'
    printf '%s\n' "Compressing the JARs requiring signing."
    printf '%s\n'
    rm -f /tmp/jars_to_sign_`date +%F`.zip
    for jar in ${jars_to_sign}
    do
      printf '%s\n'
      printf '%s\n' "Removing EBS adadmin signature from: ${jar}"
      printf '%s\n'
      zip -d "$jar" 'META-INF/*.SF' 'META-INF/*.RSA'
      printf '%s\n'
      zip /tmp/jars_to_sign_`date +%F`.zip "$jar"
    done
    printf '%s\n'
    printf '%s\n' "Script finished at: `date`"
    printf '%s\n' "Please sign the JAR files located in the archive /tmp/jars_to_sign_`date +%F`.zip"
    printf '%s\n'
  fi
else
  printf '%s\n'
  printf '%s\n' "The JAR list files are empty. Please run adadmin to regenerate the JARs with force option."
  printf '%s\n'
fi

Signing after the patching

To sign only the files modified after that patching you may modify the script to sign only the files returned by the following command after applying the patch EBS environment:

[applmgr@r122_Local ~]$ . ~/EBSapps.env patch
[applmgr@r122_Local ~]$ find $JAVA_TOP/oracle/apps/ -type f -name "*.jar" -mtime -1

This command returns the JARs modified in the last 1 day.

The adjusted script code is presented below:

printf '%s\n'
printf '%s\n' "The script support the following operations:"
printf '%s\n'
printf '%s\n' "sign - signs the JAR files right on this server using your custom command."
printf '%s\n' "copy - creates an archive with JAR files to be signed on a remote server."
printf '%s\n'
read -p "Enter the required operation: " operation
  case "$operation" in
    [sS][iI][gG][nN])
      sign_jars='Y'
      ;;
    [cC][oO][pP][yY])
      sign_jars='N'
      ;;
    *)
      printf '%s\n'
      printf '%s\n' "Invalid input. Aborting the script execution."
      printf '%s\n'
      exit 1
      ;;
  esac
adjarlist="/tmp/jars_to_sign.txt"
find $JAVA_TOP/oracle/apps/ -type f -name "*.jar" -mtime -1>"$adjarlist"
if [ -s "$adjarlist" ]; then
  jars_to_sign=`cat $adjarlist | grep '\.jar'`
  printf '%s\n'
  printf '%s\n' "Backing up the JARs to the /tmp/backup_jars_`date +%F`.zip"
  printf '%s\n'
  rm -f /tmp/backup_jars_`date +%F`.zip
  echo "$jars_to_sign"| zip -@ /tmp/backup_jars_`date +%F`.zip
 
  if [ "$sign_jars" = 'Y' ]; then
    printf '%s\n'
    printf '%s\n' "Signing the JAR files."
    printf '%s\n'
    for jar in ${jars_to_sign}
    do
      printf '%s\n'
      printf '%s\n' "Removing EBS adadmin signature from: ${jar}"
      printf '%s\n'
      zip -d "$jar" 'META-INF/*.SF' 'META-INF/*.RSA'
      printf '%s\n'
      /opt/proCertumSmartSign/jre/bin/jarsigner -keystore NONE -tsa "http://time.certum.pl" -certchain "/home/applmgr/jarsigning/bundle.pem" -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg "/home/applmgr/jarsigning/provider_simplysign.cfg" -storepass "12345" "${jar}" 31C6C11026D3227FB193E8BE24A2EFFC
    done
    printf '%s\n'
    printf '%s\n' "Script finished at: `date`"
    printf '%s\n'
  else
    printf '%s\n'
    printf '%s\n' "Compressing the JARs requiring signing."
    printf '%s\n'
    rm -f /tmp/jars_to_sign_`date +%F`.zip
    for jar in ${jars_to_sign}
    do
      printf '%s\n'
      printf '%s\n' "Removing EBS adadmin signature from: ${jar}"
      printf '%s\n'
      zip -d "$jar" 'META-INF/*.SF' 'META-INF/*.RSA'
      printf '%s\n'
      zip /tmp/jars_to_sign_`date +%F`.zip "$jar"
    done
    printf '%s\n'
    printf '%s\n' "Script finished at: `date`"
    printf '%s\n' "Please sign the JAR files located in the archive /tmp/jars_to_sign_`date +%F`.zip"
    printf '%s\n'
  fi
else
  printf '%s\n'
  printf '%s\n' "The JAR list files are empty. Please run adadmin to regenerate the JARs with force option."
  printf '%s\n'
fi