Tag Archives: Microsoft

download elasticsearch error during Open edX installation

Bumped to this error a couple of times during my install of open edX on Microsoft Azure (check out the step-by-step here).

TASK: [elasticsearch | download elasticsearch] ******************************** failed: [localhost] => {“failed”: true, “item”: “”}
msg: Failed to validate the SSL certificate for download.elasticsearch.org:443. Use validate_certs=no or make sure your managed systems have a valid CA certificate installed. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible
FATAL: all hosts have already failed — aborting

Seems like quite a few folks had it as well. One way to fix it is to set validate_certs to no in the configuration/playbooks/roles/elasticsearch/tasks/main.yml file. While I can see that the PR has been merged, I do not understand why this error still occur to me even after deleting the configuration and cloning the configuration manually.

sudo rm -rf /var/tmp/configuration
cd /var/tmp
git clone -b release https://github.com/edx/configuration
cd /var/tmp/configuration
sudo pip install -r requirements.txt
cd /var/tmp/configuration/playbooks && sudo ansible-playbook -c local ./edx_sandbox.yml -i "localhost,"

In any case, I found that my copy of configuration/playbooks/roles/elasticsearch/tasks/main.yml still does not have validate_certs=no. In order to fix that, we need to edit the file in the server.

Here are some useful commands to edit a file:

  1. For editing 
    vi main.yml
  2. For inserting 
    Press i //Do required editing
  3. For exiting 
    Press Esc
  :wq //for exiting and saving
  :q! //for exiting without saving

So, to solve our issue:

  1. Run the following code 
    cd /var/tmp/configuration/playbooks/roles/elasticsearch/tasks
vi main.yml
  2. Go to line 37, right after force=no, and press i to insert.
  3. Type validate_certs=no in a new line.
    elasticsearch1
  4. Press Esc and type :wq.
  5. Run the following command to install open edX again. 
    cd /var/tmp/configuration/playbooks && sudo ansible-playbook -c local ./edx_sandbox.yml -i "localhost,"
  6. The download elasticsearch task should not throw an error now 🙂

installing Open edX on Microsoft Azure

Few months back, my boss installed Open edX on Microsoft Azure and document his notes here. Using his notes, I managed to setup my own copy of Open edX on my Azure account and avoid most of the pitfalls (Thanks Tim!). While the whole process should be quite straightforward, it might not be if you are not familiar with Linux, Azure, SSL etc, so I decided to create a step-by-step procedure here.

What you need

  1. Azure account. This is where we host the Linux VM (we are using IaaS in this method) for the Open edX server. Sign up for trial here, if you don’t have one already.
  2. SSH client. This is used to terminal into the server. We’ll use PuTTY (putty.exe, download here).
  3. Private/public key pair. Since we’ll be using a password less VM, this is used to authenticate with the server. We’ll use openssl.exe to create a private key (openssh key) and a certificate (.pem). Openssl.exe is installed as part of Git (download here). Otherwise look at this article on how to get openssl on Windows. We’ll also use PuTTY Key Generator (puttygen.exe, download here) to create a PPK for PuTTY from the openssh key.

Once you have all you need, we’ll start the process by creating the public/private keypair to be used to authenticate to the server.

Create a public/private keypair

  1. Create a certificate with open ssl
  2. Open Command Prompt with Admin privilege.
  3. In the Command Prompt, type: 
    "\Program Files (x86)\Git\bin\openssl.exe" req -x509 -nodes -days 365 -newkey rsa:2048 -keyout myPrivateKey.key -out myCert.pem -config "c:\Program Files (x86)\Git\ssl\openssl.cnf"

    openedX1

  4. Answer the questions that are asked.
  5. It would have created two files: myPrivateKey.key and myCert.pem. More documentation on this here.
  6. Translate the private key created earlier to an RSA private key that puttygen can understand.
  7. In the Command Prompt, type: 
    "C:\Program Files (x86)\Git\bin\openssl.exe" rsa -in myPrivateKey.key -out myPrivateRSAkey.rsakey
  8. The command above should produce a new private key called myPrivateRSAkey.rsakey.
  9. Run puttygen.exe
  10. Click Load.
  11. Find your private key, which we named myPrivateRSAkey.rsakey above. You will need to change the file filter to show All Files (*.*)
  12. Click Open and click OK.
  13. Click Save Private Key and save the file as MyPrivateKey.ppk. More documentation on this here.

Provision a Linux VM on Azure

  1. Open the azure portal at https://manage.windowsazure.com and log on with your credentials
  2. Click New, select Compute, select Virtual Machine, and click From Gallery.
  3. Select Ubuntu, select Ubuntu Server 12.04 LTS, and click the next arrow. openedX2
  4. Name the Virtual Machine, select at least A2 for the size, and browse the certificate file we created earlier (myCert.pem). openedX3
  5. Ensure the Provide A Password checkbox is cleared and click the next arrow.
  6. Name your Cloud Service DNS, select your region, open the port 80 and 18010 for edX portal and edX studio respectively, and click the next arrow. openedX4
  7. Click the complete arrow and wait until the VM is completely provisioned and running.

Configure the server

  1. Run putty.exe.
  2. Fill in the host name using the Cloud Service DNS name. openedX5
  3. Click Connection, click SSH and click Auth.
  4. Browse to the private key (MyPrivateKey.ppk) and click Open. openedX6 More documentation on this here.
  5. In the terminal that opens, log on as the user created for the server.
  6. Type the following command in the terminal to update Ubuntu packages: 
    sudo apt-get update -y
sudo apt-get upgrade -y
  7. Type the following command to reboot the server: 
    sudo reboot

Install Open edX

  1. Reconnect to the server using putty.exe.
  2. Create a screen session by typing the following command in the terminal
  3. screen –d -RR
  4. Run the one step installation script
  5. The script requires that the running user can run commands as root via sudo. 
    wget https://raw.githubusercontent.com/edx/configuration/master/util/install/vagrant.sh -O - | bash
  6. If you want to install a named release, such as Birch, just set the OPENEDX_RELEASE variable before running the script. For example: 
    export OPENEDX_RELEASE=named-release/birch
wget https://raw.githubusercontent.com/edx/configuration/$OPENEDX_RELEASE/util/install/vagrant.sh -O - | bash
  7. More documentation on this here.
  8. Congratulations! Now you have the edX platform running on port 80 (http) and the edX studio on the port 18010. openedX7
  9. See edX-Managing-the-Full-Stack for how to manage and update the server once it is running.

NOTE: On my 2 counts of open edX installation, I bumped to this error.

TASK: [elasticsearch | download elasticsearch] ********************************
 failed: [localhost] => {"failed": true, "item": ""}
 msg: Failed to validate the SSL certificate for download.elasticsearch.org:443. Use validate_certs=no or make sure your managed systems have a valid CA certificate installed. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible
 FATAL: all hosts have already failed -- aborting

While the edX portal and studio seemed to be working, I did manage to install the third time without error with a little workaround. It is documented in the next post.