Creating a Self-Signed SSL Certificate to Secure Java WebSockets on Windows

So, here's the situation you're facing:

You have a Java-based server program running on a Windows computer. The computer is not a web server -- it's just a regular Windows machine -- and you want to secure its WebSocket with a self-signed SSL certificate.

Maybe you're wanting to connect in to that Java server program through an HTTPS web page. Maybe you're wanting a bit of extra security for occasional connections over the public internet. Maybe (like me) you need an SSL certificate to allow a bit of HTTPS testing in-house on a few isolated Windows PCs. Regardless, you have a Java jar running on a Windows computer, that Java jar creates a TCP WebSocket, and you want to secure that WebSocket with a self-signed SSL certificate.

What to do?

Setting a Network Name

Before starting, you will want to have a network name by which your Windows computer is known. That can be handled by either having a proper DNS name for your computer or, in a pinch, you can edit the hosts file to direct a name to the IP address of that computer. That hosts file edit is the quick hack we'll run through here.

Your hosts file can be found on your Windows computer at:

/windows/system32/drivers/etc/hosts

to edit that file, open up Notepad in Administrator mode (admin rights will be needed to save your changes) and add an entry for the IP address of interest.

For example, if you wanted to reference a computer at the local IP address of 192.168.0.3 with the DNS name mytestserver, adding the following line for that in your hosts file would direct that name to the computer:

# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
#	127.0.0.1       localhost
#	::1             localhost
192.168.0.3     mytestserver

After making that entry, a ping test from your MS DOS command prompt to that mytestserver DNS name should show you a reply from your identified IP address (if not, check that you've chosen the correct IP address for your hosts entry -- a ping will bounce off a computer running at that address).

The rest of this example assumes use of the mytestserver DNS name we hacked up in the hosts file above. For all of the following examples, where you see 'mytestserver' in a listed command, substitute in your own Windows computer's DNS name instead.

Creating a Self-Signed SSL Certificate on Windows

Now to the root of things: Creating a self-signed SSL certificate on the Windows computer itself. That can be done using the Windows Powershell utility running in Administrator mode.

The easiest way to start your Powershell is to open your Cortana search (keyboard shortcut: windows-key S), enter 'powershell' in the search prompt, and then right-click and select 'Run as administrator' on the Windows Powershell desktop app which appears in your search results.

The New-SelfSignedCertificate Windows Powershell command will create a self-signed SSL certificate and add it to your Windows certificates list.

For example, this Windows Powershell command (which is broken out on multiple lines here for clarity, but which must be entered in your Powershell on one line):

New-SelfSignedCertificate -CertStoreLocation Cert:\LocalMachine\My
 -DnsName "mytestserver"
 -FriendlyName "MyLocalTestCert"
 -NotAfter (Get-Date).AddYears(25)

Would create a new self-signed certificate for the example DNS name we're using of "mytestserver" and provide it with a human-readable name of "MyLocalTestCert". You should use your own computer's DNS name and the certificate name you would like as appropriate. That certificate would have a lifetime of 25 years (something you could safely forget about for purposes of your testing) and is ready to add to the trusted root certification authorities.

Your Windows Powershell window can be closed now -- we're done with that tool.

Making a Self-Signed SSL Certificate Trusted

With the self-signed SSL certificate created, we can now access it in the Windows Manage Computer Certificates utility. As with Windows Powershell, the easiest way to access that tool on Windows 10 is to bring up Cortana search, enter 'certificate' in the search box, and select the 'manage computer certificates' option from the list of search results.

With your Windows Certificates Manager open, open the folder branch:

"Certificates - Local Computer -> Personal -> Certificates"

You will see the MyLocalTestCert certificate you created listed in that folder.

Now, we'll want to copy that certificate to the list of trusted certificates. That can be done with a simple copy-and-paste of your certificate from the certificates manager folder:

"Certificates - Local Computer -> Personal -> Certificates"

to the folder:

"Certificates - Local Computer -> Trusted Root Certification Authorities -> Certificates"

That serves to register your created SSL certificate as one which your Windows computer will trust for SSL socket creation.

Creating a Java-Ready SSL Certificate

At this point, we have a self-signed SSL certificate created and registered as trusted on Windows. So, how do we create a Java-ready keystore?

The solution for that is the Java keytool utility, which should already be available on your computer thanks to your Java development. But before we can use the keytool, we need to export a copy of the self-signed certificate we've created.

Returning to the Windows Certificates Manager, select the certificate you imported into your Trusted Root Certification Authorities and right-click the cert to select "All Tasks -> Export...". From there, it's just a matter of following the certificate export Wizard.

In the certificate export wizard, you will want to export the private key (that key will be used in your Java keystore), use the Export File Format of "Personal Information Exchange PKCS #12 (.PFX)" and for the file export options, you will want to choose:

- Include all certificates in the certification path if possible
- Enable certificate privacy

Be sure to make note of the password you choose for your certificate -- you will be using that password later.

With your certificate data exported to file (we're using the filename myexportedcert.pfx for the example that follows), you're ready to create your Java keystore.

To provide a simple staging area for the creation of your keystore, create a temporary directory on your hard drive (say, d:\temp_work\ ) and copy your exported myexportedcert.pfx file to that directory.

With your myexportedcert.pfx file in place, open an MS DOS prompt window, change the path to your temporary directory and enter (on a single line) the keytool command:

keytool -importkeystore -srckeystore myexportedcert.pfx -srcstoretype pkcs12
-destkeystore myKeystore.jks -deststoretype JKS

and then follow the on-screen prompts to enter the password you used for the creation of your certificate.

Once all prompts have been finished, you will find a new file, myKeystore.jks, in your temporary directory. That myKeystore.jks file provides all of the detail you require to load and use your self-signed SSL certificate to secure your Java program's WebSocket.

Using Your Self-Signed SSL Certificate on Other Computers

What we'll be wanting to do next is allow your self-signed SSL certificate to be used from other computers.

Since your SSL certificate is self-signed, it will not be recognized outside of your server computer (having copied your certificate to your "Trusted Root Certificate Authorities" provided all of the authorization you needed for that system), but you can export your certificate to different computers to allow use of the self-signed SSL certificate elsewhere.

The exported certificate file in this case will not contain your private certificate information and will be safe to pass around to other computers for your testing. To create it, open up your Windows Certificates Manager, select the self-signed SSL certificate you placed in your Trusted Root Certification Authorities earlier, and right-click the certificate to open up your certificate export wizard.

This time, you will not want to export the private key for the certificate, and you will select the export file format of "DER encoded binary X.509 (.CER)". Once you have completed the certificate export wizard and have exported a new .cer file, that file can be added to the trusted keys of any other computer you would like to access your SSL socket from.

To import your .cer file on another Windows computer, just open the Windows Certificates Manager on your other computer, open the folder:

"Certificates - Local Computer -> Trusted Root Certification Authorities -> Certificates"

and import your created .cer file. From there, you will be able to connect to your test computer's SSL socket.

Don't forget: If you do not have a DNS name specified for the computer hosting your Java SSL socket server program, you will need to add an entry to that computer's hosts file so that it will recognize the hostname tied to your SSL certificate (for example, the mytestserver DNS name we used earlier in our quick hosts file hack).

That's it! You have created a self-signed SSL certificate which can be used to secure a Java TCP socket for testing, and you can now access that certificate from whichever computer you would find helpful.

This is a rather niche case for SSL certificates and, for a permanent solution, you will want to purchase a certificate signed by an external certificate authority rather than work with a self-signed cert on your systems. But these particular steps were helpful for my own testing, and I hope the details outlined here will help save time in your own troubleshooting and development.