Introduction
Crate.IO (Crate) is an elastic distributed SQL data store. It combines a SQL interface with the document-oriented approach of a modern NoSQL database. Crate is built from a number of different technologies including Java, Facebook’s Presto, Elasticsearch, Apache Lucene, and Netty. Further details can be found in the Wikipedia Entry for CrateIO.
Requirements
- A ProfitBricks account and access to the DCD (Data Center Designer)
Installation
First create a new data center or use one that is already provisioned. Then add a composite instance with at least one CPU core and some memory and hard disk space. In the Image drop-down menu, select the ProfitBricks Images menu and then choose Debian-8-server-2016-03-01.
You will need to add and/or connect Internet Access to the data center, and make sure to connect “NIC 0” to it.
It should end up looking something like this:
Go ahead and provision the changes using the large button in the lower right corner of the DCD.
Once provisioning is complete, log in as root via ssh to the newly-provisioned server. You may utilize the “Remote Console” inside the DCD if you prefer. Since this is the first time we have connected to the server, we will be forced to change the root password.
login as: root
[email protected][your-server-ip-here]'s password:
You are required to change your password immediately (root enforced)
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Changing password for root.
(current) UNIX password:
Enter new UNIX password:
Retype new UNIX password:
[email protected]:~#
Add a User Account
Our newly-provisioned server does not have any users yet, so we will add one:
[email protected]:~# adduser tutorial
Adding user `tutorial' ...
Adding new group `tutorial' (1000) ...
Adding new user `tutorial' (1000) with group `tutorial' ...
Creating home directory `/home/tutorial' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for tutorial
Enter the new value, or press ENTER for the default
Full Name []: Crate Tutorial
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] Y
Debian 8 does not have sudo available by default. Let’s update the system using apt-get update and apt-get dist-upgrade:
[email protected]:~# apt-get update
Ign http://ftp.us.debian.org jessie InRelease
Get:1 http://ftp.us.debian.org jessie-updates InRelease [142 kB]
[some-output-removed]
Fetched 680 kB in 4s (158 kB/s)
Reading package lists... Done
[email protected]:~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
bind9-host dnsutils libbind9-90 libdns-export100 libdns100 libirs-export91 libisc-export95 libisc95
libisccc90 libisccfg-export90 libisccfg90 liblwres90 libssl1.0.0 linux-image-3.16.0-4-amd64 openssl perl
perl-base perl-modules
18 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 43.8 MB of archives.
After this operation, 7,168 B of additional disk space will be used.
Do you want to continue? [Y/n]
Answer with Y and the upgrade process will continue. Once it completes, we will go ahead and install sudo using apt-get install sudo:
[email protected]:~# apt-get install sudo
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
sudo
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 855 kB of archives.
After this operation, 2,390 kB of additional disk space will be used.
Get:1 http://ftp.us.debian.org/debian/ jessie/main sudo amd64 1.8.10p3-1+deb8u3 [855 kB]
Fetched 855 kB in 0s (1,318 kB/s)
Selecting previously unselected package sudo.
(Reading database ... 23804 files and directories currently installed.)
Preparing to unpack .../sudo_1.8.10p3-1+deb8u3_amd64.deb ...
Unpacking sudo (1.8.10p3-1+deb8u3) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for systemd (215-17+deb8u3) ...
Setting up sudo (1.8.10p3-1+deb8u3) ...
Processing triggers for systemd (215-17+deb8u3) ...
Now we can grant our new user sudo access:
[email protected]:~# adduser tutorial sudo
Adding user `tutorial' to group `sudo' ...
Adding user tutorial to group sudo
Done.
Those tasks are complete, so we will reboot the server and log in as our newly-created user instead of root.
Add the Crate Repository
The Crate Team has packages available for recent Debian releases. This tutorial is focused on Debian 8 so we will proceed by configuring the appropriate repository on the server. Based on the information found at Crate.io Debian Jessie Installation, we need to make sure some key packages are installed first:
[email protected]:~$ sudo apt-get install apt-transport-https
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
[sudo] password for tutorial:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libcurl3-gnutls libldap-2.4-2 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1
Suggested packages:
libsasl2-modules-otp libsasl2-modules-ldap libsasl2-modules-sql libsasl2-modules-gssapi-mit libsasl2-modules-gssapi-heimdal
The following NEW packages will be installed:
apt-transport-https libcurl3-gnutls libldap-2.4-2 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,065 kB of archives.
After this operation, 2,194 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Enter Y to continue the package installation. Once it completes, we can verify that apt-transport-https was installed successfully:
[email protected]:~$ dpkg --list |grep apt-transport-https
ii apt-transport-https 1.0.9.8.2 amd64 https download transport for APT
Now we can proceed to get the GPG key using wget:
[email protected]:~$ wget https://cdn.crate.io/downloads/apt/DEB-GPG-KEY-crate
--[date]-- https://cdn.crate.io/downloads/apt/DEB-GPG-KEY-crate
Resolving cdn.crate.io (cdn.crate.io)... 199.27.79.207
Connecting to cdn.crate.io (cdn.crate.io)|199.27.79.207|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3150 (3.1K) [application/octet-stream]
Saving to: ‘DEB-GPG-KEY-crate’ DEB-GPG-KEY-crate
100%[=========================================>] 3.08K --.-KB/s in 0s
[date] (53.0 MB/s) - ‘DEB-GPG-KEY-crate’ saved [3150/3150]
Add the GPG key by running sudo apt-key add DEB-GPG-KEY-crate:
[email protected]:~$ sudo apt-key add DEB-GPG-KEY-crate
OK
We can verify it was added by taking a look at the output of apt-key list:
[email protected]:~$ sudo apt-key list
/etc/apt/trusted.gpg
--------------------
pub 4096R/06F6EAEB 2014-01-07
uid CRATE Jenkins <[email protected]>
sub 4096R/0EFD9B38 2014-01-07
We will proceed to add the Crate repositories. The default /etc/apt/sources.list will look something like this:
[email protected]:~$ sudo cat /etc/apt/sources.list
[sudo] password for tutorial:
# deb http://ftp.us.debian.org/debian jessie main
deb http://ftp.us.debian.org/debian jessie main
deb-src http://ftp.us.debian.org/debian jessie main
deb http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main
# jessie-updates, previously known as 'volatile'
deb http://ftp.us.debian.org/debian jessie-updates main
deb-src http://ftp.us.debian.org/debian jessie-updates main
Run the following command to add the Crate repositories:
sudo su -c "echo 'deb https://cdn.crate.io/downloads/apt/stable/ jessie main
deb-src https://cdn.crate.io/downloads/apt/stable/ jessie main' >> /etc/apt/sources.list"
Now our /etc/apt/sources.list will have two new lines at the end:
[email protected]:~$ sudo cat /etc/apt/sources.list
# deb http://ftp.us.debian.org/debian jessie main
deb http://ftp.us.debian.org/debian jessie main
deb-src http://ftp.us.debian.org/debian jessie main
deb http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main
# jessie-updates, previously known as 'volatile'
deb http://ftp.us.debian.org/debian jessie-updates main
deb-src http://ftp.us.debian.org/debian jessie-updates main
deb https://cdn.crate.io/downloads/apt/stable/ jessie main
deb-src https://cdn.crate.io/downloads/apt/stable/ jessie main
Since that completed successfully, we will update the list of available packages:
sudo apt-get update
Install Crate
Next we will run sudo apt-get install crate:
[email protected]:~$ sudo apt-get install crate
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
crate : Depends: java8-runtime-headless but it is not installable or
java8-runtime but it is not installable
E: Unable to correct problems, you have held broken packages.
Well, it did not work due to “unmet dependencies”. We can get access to “java8” by adding the “jessie-backports” repository:
sudo su -c "echo 'deb http://http.debian.net/debian jessie-backports main' >> /etc/apt/sources.list"
and update once again:
sudo apt-get update
and install openjdk-8-jre-headless:
[email protected]:~$ sudo apt-get install openjdk-8-jre-headless
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
ca-certificates-java dbus fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcap-ng0 libcups2 libdbus-1-3 libfontconfig1 libjpeg62-turbo liblcms2-2 libnspr4 libnss3 libpcsclite1 libxi6 libxrender1 libxtst6 x11-common
Suggested packages:
dbus-x11 default-jre equivs cups-common liblcms2-utils pcscd openjdk-8-jre-jamvm libnss-mdns fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho ttf-wqy-microhei ttf-wqy-zenhei fonts-indic
The following NEW packages will be installed:
ca-certificates-java dbus fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcap-ng0 libcups2 libdbus-1-3 libfontconfig1 libjpeg62-turbo liblcms2-2 libnspr4 libnss3 libpcsclite1 libxi6 libxrender1 libxtst6 openjdk-8-jre-headless x11-common
0 upgraded, 22 newly installed, 0 to remove and 1 not upgraded.
Need to get 31.6 MB of archives.
After this operation, 113 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Answering Y will install the necessary packages. Once it completes, we will try to install Crate again:
[email protected]:~$ sudo apt-get install crate
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
crate
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 43.7 MB of archives.
After this operation, 53.8 MB of additional disk space will be used.
Get:1 https://cdn.crate.io/downloads/apt/stable/ jessie/main crate all 0.54.7-1~jessie [43.7 MB]
Fetched 43.7 MB in 10s (4,117 kB/s)
Selecting previously unselected package crate.
(Reading database ... 24586 files and directories currently installed.)
Preparing to unpack .../crate_0.54.7-1~jessie_all.deb ...
Unpacking crate (0.54.7-1~jessie) ...
Processing triggers for systemd (215-17+deb8u3) ...
Setting up crate (0.54.7-1~jessie) ...
Processing triggers for systemd (215-17+deb8u3) ...
It appears to have installed successfully and should be started automatically. Let’s confirm that it is running by looking for it in the output of ps.
[email protected]:~$ ps wwwaux |grep java
crate 37133 14.6 11.4 3064736 235204 ? Ssl 21:28 0:08 java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8 -Dcrate -Des.path.home=/usr/share/crate -Des.config=/usr/share/crate/config/crate.yml -cp /usr/share/crate/lib/crate-app-0.54.7.jar:/usr/share/crate/lib/*:/usr/share/crate/lib/sigar/* -Des.config=/usr/share/crate/config/crate.yml -Des.path.home=/usr/share/crate -Des.path.conf=/usr/share/crate/config -Des.path.logs=/var/log/crate io.crate.bootstrap.CrateF
FYI – we can stop and start Crate using the service command:
[email protected]:~$ sudo service crate stop
[email protected]:~$ sudo service crate start
Configure a Firewall
Let’s stop the server and get a firewall set up to provide some protection from unwanted visitors.
[email protected]:~$ sudo service crate stop
Debian 8 has Iptables available as a host-based firewall. It is not enabled by default. You are welcome to use Iptables to protect your server. We also have the option to configure the ProfitBricks data center firewall using the DCD. For this tutorial we will briefly demonstrate using the ProfitBricks firewall.
We will add a few rules to restrict traffic to the server.
Use the Create Firewall rule drop-down menu to add TCP Rule twice. Port 22 is for ssh and port 4200 is for crate. You will need to substitute the appropriate Source IP and Target IP addresses into these rules. Save the new rules when you are done.
Now we Provision the changes inside the DCD to enable the firewall.
If you run into an issue accessing the server once the firewall is enabled, use the DCD to verify the rules are correct.
Exploring Crate
Now that we have limited access to the server from our IP address, we can start up the database and take a look at the web interface. Open up a browser and go to https://SERVER_IP:4200/admin and you should see the web interface.
If you would like to import some sample data, please jump over and take a look at the Install Crate.IO on Ubuntu 15 tutorial.