Month: July 2018

 

 

Striim in the Microsoft Azure Cloud enables companies to simplify real-time data movement to Azure by enabling heterogeneous data ingestion, enrichment, and transformation in a single solution before it delivers the data with sub-second latency. Brought to you by the core team behind GoldenGate Software, Striim offers a non-intrusive, quick-to-deploy, and easy-to-iterate solution for streaming data movement to Azure.

It’s easy to get started with Striim in the Azure Cloud. We offer Azure Marketplace solutions for Azure SQL Data Warehouse, Azure Cosmos DB, SQL Database, Azure Database for PostgreSQL, Azure Storage, and HDInsight. However, if you want continuous data movement to Azure Database for MySQL or other Azure services, you can quickly set up your own instance of Striim on a VM.

This quick-start guide assumes you already have an Azure account set up, and should take about 20 minutes to complete.

 

Create Ubuntu VM on Microsoft Azure

1. Open up your favorite browser and navigate to Azure’s dashboard at portal.azure.com
   
2. Click on the + Create a resource button on the top of the left hand menu

3. Search for Ubuntu and select Ubuntu Server 16.04 LTS. This version is certified to work with Striim out of the box.

4. Select Create

5. Enter the information for your VM. For this demo, I’m using a password as authentication type. For a more secure connection for production workloads, select SSH public key and generate it through terminal if you’re on a Mac. Click OK when you’re done entering the connection information.

6. Choose a size for your VM. I’m using D2s_v3 for this demo. However, choose a larger size if need be for production workloads. Note that you can always scale the VM after creating it if you find your initial choice is not sufficient. Press Select to continue.

7. Configure the additional Settings of the VM. If you need high availability for your configuration, select it here and specify your Availability set. As I’m only using this VM for demo purposes and don’t need high availability, I’ll skip it.

8. The important piece here is to open up the specific ports that you need to access on the VM. Make sure to open HTTP and SSH, as we’ll need them for connecting to the VM and Striim. Select OK when you’re done.

9. Azure will validate your Ubuntu VM, and make sure everything is correct. Make sure everything looks good on your end as well, and select Create.

10. It may take awhile to deploy the VM, so just be patient until it is deployed. When the VM is properly deployed, it will show up under All resources in your Azure Dashboard

 

1. Navigate to your Azure Dashboard and select the Ubuntu VM you just created.

2. First, ensure the correct ports are open in the Networking pane on the Azure portal (Striim needs to have the following ports open: 1527, 5701, 9080, 9300, 49152-65535, and 54237). You can find more information about the required ports in Installation and Configuration Guide of Striim’s documentation.

3. Select Connect in the top middle of the screen and copy the Login using VM local account address.

4. Note: these instructions are for a Mac. Open up a new Terminal window and paste the SSH command you copied earlier from the Azure portal. There is a slightly different process on Windows to SSH into a VM, and a quick Google search should get you started with PuTTY or other tools.

5. Type yes to continue, and enter the password you created through the Azure portal

6. Congratulations! Now you’re logged in to the VM you created on Azure. From here, first we’ll install Java, and then download and install Striim.

7. Following the instructions here: https://medium.com/coderscorner/installing-oracle-java-8-in-ubuntu-16-10-845507b13343, install Oracle’s JDK 8. First, add Oracle’s PPA to your list of sources, pressing ENTER when necessary:

– sudo add-apt-repository ppa:webupd8team/java

8. Update your package repository, typing Y or yes when necessary

– sudo apt-get update

9. Install Java. There will be two screens that pop up during the installation process that require you to accept Oracle’s license terms

– sudo apt-get install oracle-java8-installer

10. To make sure Java installed correctly, type java -version to ensure that Oracle JDK 1.8.0 is installed.

11. Now that Java is installed, we can install Striim on your Azure VM. First, download Derby using wget.

– sudo su
– wget https://s3-us-west-1.amazonaws.com/striim-downloads/Releases/3.8.3A/striim-dbms-3.8.3A-Linux.deb

12. Download Striim

– wget https://s3-us-west-1.amazonaws.com/striim-downloads/Releases/3.8.3A/striim-node-3.8.3A-Linux.deb

13. Now, install both the Striim and Derby packages

dpkg -i striim-node-3.8.3A-Linux.deb
dpkg -i striim-dbms-3.8.3A-Linux.deb

14. Edit the /opt/striim/conf/striim.conffile using your favorite text editor, and enter the following fields at the top of the file:
– WA_CLUSTER_NAME: choose a unique name for the new cluster (unique in the sense that it is not already used by any existing Striim cluster on the same network)
– WA_CLUSTER_PASSWORD: will be used by other servers to connect to the cluster and for other cluster-level operations
– WA_ADMIN_PASSWORD: will be assigned to Striim’s default admin user account
– WA_IP_ADDRESS: the IP address of this server to be used by Striim
– WA_PRODUCT_KEY and WA_LICENSE_KEY: If you have keys, specify them, otherwise leave blank to run Striim on a 30-day trial license.
– NOTE: You cannot create a multi-server cluster using a trial license.
– WA_COMPANY_NAME: If you specified keys, this must exactly match the associated company name. Otherwise, enter your company name.

15. If using Vi, execute the following commands:

– vi /opt/striim/conf/striim.conf
– This edits the existing file
– I
– Type/copy and paste your properties
– Press esc key
– Type : x to save file

16. Now, we can go ahead and start up Striim as a process. Execute the following commands:
– systemctl enable striim-dbms
– systemctl start striim-dbms
– Wait ten seconds, then
– systemctl enable striim-node
– systemctl start striim-node

17. Enter the following command and wait for a message similar to: Please go to … to administer, or use console.
– tail -F /opt/striim/logs/striim-node.log

18. Finally, navigate to Striim’s Web UI located at <VM Public IP Address>:9080 and login with the information you provided during the setup process. If you’re new to Striim, we recommend you go through the quickstart located under the ? Help > Documentation button in the upper right hand corner of the UI.

You are now ready to enable continuous data movement to Azure. For technical assistance, please feel free to contact support@striim.com. For a demo of Striim’s full capabilities around moving data to Azure, please schedule a demo with one of our lead technologists.

 

 

Out with the old and in with the new: Streaming data integration offers so much more than traditional logical replication – from cloud integration, to advanced analytics, to support for machine learning.

There was once a time, not too long ago, when only data in databases were collected and analyzed. This sounds like a crazy concept today based on the fact that now there are a myriad of digital frameworks for data to reside in – log files in machines, connected devices, Hadoop, Kafka, cloud-based data stores and applications etc. This is due, in part, to the wide variety of sources where data originates, not to mention the insane amount of data now being created. In its day, logical replication was the best option for organizations to share data across systems with low-latency so that companies were working with the most up-to-date data possible, regardless of location.

However, over the last few years, thanks to digital transformation, we’ve seen a fundamental shift in data management, demand for faster and better analytics, and advancements in computing technologies. We’ve seen CPU and RAM get cheaper and faster, enabling organizations to ingest, process, and analyze broader types of data, in real time, regardless of what environment enterprise data is in.

While logical replication (also known as transactional replication) vendors have done their best to support integration with modern data sources and targets, they weren’t designed to reliably and securely stream high-velocity data across new IoT, advanced analytics, and cloud systems. As a result, companies who try to use logical replication systems for next-generation analytics solutions, whether on-premises or in the cloud, often feel like they’re fitting round pegs in square holes.

The Striim platform was built from the ground up with a streaming architecture in mind, offering capabilities that far exceed where transactional replication falls short to bring enterprise companies to the evolutionary next stage of a modern data architecture.

The image below succinctly details the differences between logical replication and streaming data integration, and why Striim is the better option for implementing a streaming architecture to gain maximum value from real-time data.

Companies need to work with all of their data, while it’s still relevant, in order to gain data-driven insights. A streaming architecture helps digital businesses make the most of their data assets for operational excellence, and for that, it needs solutions that go beyond just real-time data movement between databases. Companies can start building a streaming architecture with platforms like Striim that make it easy to collect, prepare, analyze, and visualize high-volume, high-velocity data (structured, semi-structured, or unstructured) from diverse set of sourcesin real time, and share with any system regardless of its location.

Learn more about how your organization can take the first step in adopting a streaming architecture by visiting the Striim website, where you can find further information about our platform’s capabilities, use cases, case studies, and other materials to guide you in the right direction. Additionally, you can download the Striim platform or schedule a demo to learn more.