Install Crate.IO on Ubuntu 15

By | October 29, 2016

Introduction

Crate.IO (Crate) is an elastic distributed SQL data store. It combines a SQL interface with the document-oriented approach of a modern NoSQL database. Crate is built from a number of different technologies including Java, Facebook’s Presto, Elasticsearch, Apache Lucene, and Netty. Further details can be found in the Wikipedia Entry for CrateIO.

Requirements

  • ProfitBricks account and access to the DCD (Data Center Designer)

Installation

First create a new data center or use one that is already provisioned. Then add a composite instance with at least one CPU core and some memory and hard disk space. In the Image drop-down menu, select the ProfitBricks Images menu and then choose Ubuntu-15.10-server-2016-03-01.

ProfitBricks Ubuntu 15.10 Server Image

You will also want to add Internet Access to the data center, and connect “NIC 0” to it.

It should end up looking something like this:

Example DCD layout for Ubuntu 15.10 Server Crate Tutorial

Go ahead and provision the changes using the large button in the lower right corner of the DCD.

Once provisioning is complete, log in as root via ssh to the newly-provisioned server. You may utilize the “Remote Console” inside the DCD if you prefer. Since this is the first time we have connected to the server, we will be forced to change the root password.

Add a User Account

Our newly-provisioned server does not have any users yet, so we will add one:

[email protected]:~# adduser tutorial
Adding user `tutorial' ...
Adding new group `tutorial' (1000) ...
Adding new user `tutorial' (1000) with group `tutorial' ...
Creating home directory `/home/tutorial' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for tutorial
Enter the new value, or press ENTER for the default
        Full Name []: Tutorial User
        Room Number []:
        Work Phone []:
        Home Phone []:
        Other []:
Is the information correct? [Y/n] Y

We want to use this account to manage the system, so we will grant it sudo access:

[email protected]:~# adduser tutorial sudo
Adding user `tutorial' to group `sudo' ...
Adding user tutorial to group sudo
Done.

Log out and back in as the new user to confirm that it is working properly.

Update Ubuntu

It is a good idea to keep the system up to date, so run:

sudo apt-get update

followed by:

sudo apt-get dist-upgrade

Once the updates are installed, reboot the server so that the newly-installed kernel will be loaded.

Add the Crate Repository

The Crate Team has packages available for recent Ubuntu releases. This tutorial is focused on Ubuntu 15.10 server so we will proceed by configuring the appropriate repository on the server. Based on the information found at Crate.io Ubuntu Installation, we need to make sure some key packages are installed:

[email protected]:~$ dpkg --get-selections | grep software
libsoftware-license-perl                        install
python3-software-properties                     install
software-properties-common                      install

Both software-properties-common and python3-software-properties are present, so we will move on and run sudo add-apt-repository ppa:crate/stable which should result in output similar to this:

[email protected]:~$ sudo add-apt-repository ppa:crate/stable
[sudo] password for tutorial:
 CRATE Packages for Ubuntu
 More info: https://launchpad.net/~crate/+archive/ubuntu/stable
Press [ENTER] to continue or ctrl-c to cancel adding it

gpg: keyring `/tmp/tmpalmz4cmn/secring.gpg' created
gpg: keyring `/tmp/tmpalmz4cmn/pubring.gpg' created
gpg: requesting key 3D089FBC from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpalmz4cmn/trustdb.gpg: trustdb created
gpg: key 3D089FBC: public key "Launchpad PPA for CRATE" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK

Since that completed successfully, we will update the list of available packages:

sudo apt-get update

Install Crate

Next we will run sudo apt-get install crate:

[email protected]:~$ sudo apt-get install crate
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  ca-certificates-java fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libfontconfig1 libjpeg-turbo8
  libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpython-stdlib libxi6 libxrender1 libxtst6 openjdk-8-jre-headless python python-minimal python2.7
  python2.7-minimal x11-common
Suggested packages:
  default-jre equivs cups-common liblcms2-utils openjdk-8-jre-jamvm libnss-mdns fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho ttf-wqy-microhei
  ttf-wqy-zenhei fonts-indic python-doc python-tk python2.7-doc binutils binfmt-support
Recommended packages:
  python:any
The following NEW packages will be installed:
  ca-certificates-java crate fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libfontconfig1
  libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpython-stdlib libxi6 libxrender1 libxtst6 openjdk-8-jre-headless python python-minimal python2.7
  python2.7-minimal x11-common
0 upgraded, 26 newly installed, 0 to remove and 0 not upgraded.
Need to get 75.5 MB of archives.
After this operation, 169 MB of additional disk space will be used.
Do you want to continue? [Y/n]

Answering Y will allow the installation to proceed and install a number of packages.

We can stop and start crate using the service command:

[email protected]:~$ sudo service crate stop

[email protected]:~$ sudo service crate start

And confirm that it is running by looking for it in the output of ps.

[email protected]:~$ ps waux |grep crate
crate     4209 86.3  7.1 3030784 146068 ?      Ssl  22:37   0:02 java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8 -Dcrate -Des.path.home=/usr/share/crate -Des.config=/usr/share/crate/config/crate.yml -cp /usr/share/crate/lib/crate-app-0.54.7.jar:/usr/share/crate/lib/*:/usr/share/crate/lib/sigar/* -Des.path.home=/usr/share/crate -Des.path.conf=/etc/crate -Des.config=/etc/crate/crate.yml -Des.path.logs=/var/log/crate io.crate.bootstrap.CrateF
tutorial  4228  0.0  0.1  11796  2060 pts/0    S+   22:37   0:00 grep --color=auto crate

Configure a Firewall

Let’s stop the server and get a firewall setup to provide some protection from unwanted visitors.

[email protected]:~$ sudo service crate stop

Ubuntu provides the “Uncomplicated Firewall” tool, ufw, to manage iptables directly on the server. We also have the option to configure the ProfitBricks data center firewall using the DCD. For this tutorial we will use ufw. First we will check the status:

[email protected]:~$ sudo ufw status verbose
Status: inactive

We will add two rules to restrict traffic to the server. One for ssh traffic on port 22, and one for crate traffic on port 4200. You will need to substitute the IP address that you are connecting from into these rules:

[email protected]:~$ sudo ufw allow proto tcp from [your-ip-address-here] to any port 22
Rules updated
[email protected]:~$ sudo ufw allow proto tcp from [your-ip-address-here] to any port 4200
Rules updated

Now we enable the firewall:

[email protected]:~$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

and verify the status:

[email protected]:~$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    [your-ip-address-here]
4200/tcp                   ALLOW IN    [your-ip-address-here]

If you run into an issue accessing the server after the firewall is enabled, then use the “Remote Console” inside the DCD to disable it with the command sudo ufw disable and verify that the rules are correct.

Crate Admin Web Interface

Now that we have limited access to the server from our IP address, we can start up the database and take a look at the web interface. Open up a browser and go to https://SERVER_IP:4200/admin and you should see:

Crate Web Interface

Import Sample Data

We did not step through the process of importing sample data from Twitter in the Get Started with Crate.IO on CentOS 7 tutorial, we will go ahead and do that now.

Selecting the i icon in the left-navigation section of the admin interface brings us to this screen:

Import tweets for testing

Pressing the Import tweets for testing button takes us away to an authorization page:

Import tweets for testing authorization

Once we successfully Authorize app, data begins to stream in. Go ahead and press the Enough! Stop importing tweets button after some records have come in. You can return here later and import additional tweets.

Stop importing tweets

Returning to the home screen of the admin app we now see that some sample data is available.

Imported tweets

Сomments аrchive