Crate.IO (Crate) is an elastic distributed SQL data store. It combines a SQL interface with the document-oriented approach of a modern NoSQL database. Crate is built from a number of different technologies including Java, Facebook’s Presto, Elasticsearch, Apache Lucene, and Netty. Further details can be found in the Wikipedia Entry for CrateIO.
- ProfitBricks account and access to the DCD (Data Center Designer)
First create a new data center or use one that is already provisioned. Then add a composite instance with at least one CPU core and some memory and hard disk space. In the Image drop-down menu, select the ProfitBricks Images menu and then choose Ubuntu-15.10-server-2016-03-01.
You will also want to add Internet Access to the data center, and connect “NIC 0” to it.
It should end up looking something like this:
Go ahead and provision the changes using the large button in the lower right corner of the DCD.
Once provisioning is complete, log in as
ssh to the newly-provisioned server. You may utilize the “Remote Console” inside the DCD if you prefer. Since this is the first time we have connected to the server, we will be forced to change the
Add a User Account
Our newly-provisioned server does not have any users yet, so we will add one:
[email protected]:~# adduser tutorial Adding user `tutorial' ... Adding new group `tutorial' (1000) ... Adding new user `tutorial' (1000) with group `tutorial' ... Creating home directory `/home/tutorial' ... Copying files from `/etc/skel' ... Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully Changing the user information for tutorial Enter the new value, or press ENTER for the default Full Name : Tutorial User Room Number : Work Phone : Home Phone : Other : Is the information correct? [Y/n] Y
We want to use this account to manage the system, so we will grant it
[email protected]:~# adduser tutorial sudo Adding user `tutorial' to group `sudo' ... Adding user tutorial to group sudo Done.
Log out and back in as the new user to confirm that it is working properly.
It is a good idea to keep the system up to date, so run:
sudo apt-get update
sudo apt-get dist-upgrade
Once the updates are installed, reboot the server so that the newly-installed kernel will be loaded.
Add the Crate Repository
The Crate Team has packages available for recent Ubuntu releases. This tutorial is focused on Ubuntu 15.10 server so we will proceed by configuring the appropriate repository on the server. Based on the information found at Crate.io Ubuntu Installation, we need to make sure some key packages are installed:
[email protected]:~$ dpkg --get-selections | grep software libsoftware-license-perl install python3-software-properties install software-properties-common install
python3-software-properties are present, so we will move on and run
sudo add-apt-repository ppa:crate/stable which should result in output similar to this:
[email protected]:~$ sudo add-apt-repository ppa:crate/stable [sudo] password for tutorial: CRATE Packages for Ubuntu More info: https://launchpad.net/~crate/+archive/ubuntu/stable Press [ENTER] to continue or ctrl-c to cancel adding it gpg: keyring `/tmp/tmpalmz4cmn/secring.gpg' created gpg: keyring `/tmp/tmpalmz4cmn/pubring.gpg' created gpg: requesting key 3D089FBC from hkp server keyserver.ubuntu.com gpg: /tmp/tmpalmz4cmn/trustdb.gpg: trustdb created gpg: key 3D089FBC: public key "Launchpad PPA for CRATE" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) OK
Since that completed successfully, we will update the list of available packages:
sudo apt-get update
Next we will run
sudo apt-get install crate:
[email protected]:~$ sudo apt-get install crate Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: ca-certificates-java fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libfontconfig1 libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpython-stdlib libxi6 libxrender1 libxtst6 openjdk-8-jre-headless python python-minimal python2.7 python2.7-minimal x11-common Suggested packages: default-jre equivs cups-common liblcms2-utils openjdk-8-jre-jamvm libnss-mdns fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho ttf-wqy-microhei ttf-wqy-zenhei fonts-indic python-doc python-tk python2.7-doc binutils binfmt-support Recommended packages: python:any The following NEW packages will be installed: ca-certificates-java crate fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libfontconfig1 libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpython-stdlib libxi6 libxrender1 libxtst6 openjdk-8-jre-headless python python-minimal python2.7 python2.7-minimal x11-common 0 upgraded, 26 newly installed, 0 to remove and 0 not upgraded. Need to get 75.5 MB of archives. After this operation, 169 MB of additional disk space will be used. Do you want to continue? [Y/n]
Answering Y will allow the installation to proceed and install a number of packages.
We can stop and start crate using the
[email protected]:~$ sudo service crate stop [email protected]:~$ sudo service crate start
And confirm that it is running by looking for it in the output of
[email protected]:~$ ps waux |grep crate crate 4209 86.3 7.1 3030784 146068 ? Ssl 22:37 0:02 java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8 -Dcrate -Des.path.home=/usr/share/crate -Des.config=/usr/share/crate/config/crate.yml -cp /usr/share/crate/lib/crate-app-0.54.7.jar:/usr/share/crate/lib/*:/usr/share/crate/lib/sigar/* -Des.path.home=/usr/share/crate -Des.path.conf=/etc/crate -Des.config=/etc/crate/crate.yml -Des.path.logs=/var/log/crate io.crate.bootstrap.CrateF tutorial 4228 0.0 0.1 11796 2060 pts/0 S+ 22:37 0:00 grep --color=auto crate
Configure a Firewall
Let’s stop the server and get a firewall setup to provide some protection from unwanted visitors.
[email protected]:~$ sudo service crate stop
Ubuntu provides the “Uncomplicated Firewall” tool,
ufw, to manage
iptables directly on the server. We also have the option to configure the ProfitBricks data center firewall using the DCD. For this tutorial we will use
ufw. First we will check the status:
[email protected]:~$ sudo ufw status verbose Status: inactive
We will add two rules to restrict traffic to the server. One for
ssh traffic on port 22, and one for
crate traffic on port
4200. You will need to substitute the IP address that you are connecting from into these rules:
[email protected]:~$ sudo ufw allow proto tcp from [your-ip-address-here] to any port 22 Rules updated [email protected]:~$ sudo ufw allow proto tcp from [your-ip-address-here] to any port 4200 Rules updated
Now we enable the firewall:
[email protected]:~$ sudo ufw enable Command may disrupt existing ssh connections. Proceed with operation (y|n)? y Firewall is active and enabled on system startup
and verify the status:
[email protected]:~$ sudo ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22/tcp ALLOW IN [your-ip-address-here] 4200/tcp ALLOW IN [your-ip-address-here]
If you run into an issue accessing the server after the firewall is enabled, then use the “Remote Console” inside the DCD to disable it with the command
sudo ufw disable and verify that the rules are correct.
Crate Admin Web Interface
Now that we have limited access to the server from our IP address, we can start up the database and take a look at the web interface. Open up a browser and go to
https://SERVER_IP:4200/admin and you should see:
Import Sample Data
We did not step through the process of importing sample data from Twitter in the Get Started with Crate.IO on CentOS 7 tutorial, we will go ahead and do that now.
Selecting the i icon in the left-navigation section of the admin interface brings us to this screen:
Pressing the Import tweets for testing button takes us away to an authorization page:
Once we successfully Authorize app, data begins to stream in. Go ahead and press the Enough! Stop importing tweets button after some records have come in. You can return here later and import additional tweets.
Returning to the home screen of the admin app we now see that some sample data is available.