Oracle R Enterprise installation

In this post I would like to install Oracle R Enterprise 1.4.0. version.

Note: I am going to install the tool (or option) on a linux server (OEL 6.3 64 bit) with Oracle DB 11.2.0.4., so only "server" configuration is described.

Step 1 - Oracle R 3.0.1

Let's start. According to documentation, Oracle R 3.0.1 is required for Oracle R Enterprise 1.4.0 (I think you can use the regular R, but Oracle strongly recommends to use their own version). Install the software using yum.

[root@oel ~]# yum install R-3.0.1
Loaded plugins: refresh-packagekit, security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package R.x86_64 0:3.0.1-2.el6 will be installed
--> Processing Dependency: libRmath-devel = 3.0.1-2.el6 for package: R-3.0.1-2.el6.x86_64
--> Processing Dependency: R-devel = 3.0.1-2.el6 for package: R-3.0.1-2.el6.x86_64
--> Running transaction check
---> Package R-devel.x86_64 0:3.0.1-2.el6 will be installed
--> Processing Dependency: R-core = 3.0.1-2.el6 for package: R-devel-3.0.1-2.el6.x86_64
---> Package libRmath-devel.x86_64 0:3.0.1-2.el6 will be installed
--> Processing Dependency: libRmath = 3.0.1-2.el6 for package: libRmath-devel-3.0.1-2.el6.x86_64
--> Running transaction check
---> Package R-core.x86_64 0:3.0.1-2.el6 will be installed
---> Package libRmath.x86_64 0:3.0.1-2.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                                     Arch                                Version                                  Repository                               Size
========================================================================================================================================================================
Installing:
 R                                           x86_64                              3.0.1-2.el6                              ol6_addons                               15 k
Installing for dependencies:
 R-core                                      x86_64                              3.0.1-2.el6                              ol6_addons                               45 M
 R-devel                                     x86_64                              3.0.1-2.el6                              ol6_addons                               86 k
 libRmath                                    x86_64                              3.0.1-2.el6                              ol6_addons                              111 k
 libRmath-devel                              x86_64                              3.0.1-2.el6                              ol6_addons                               20 k

Transaction Summary
========================================================================================================================================================================
Install       5 Package(s)

Total download size: 46 M
Installed size: 81 M
Is this ok [y/N]: y
Downloading Packages
Rdevel-3.0.1-2.el6.x86_64.rpm                                                                                                                   |  86 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : libRmath-3.0.1-2.el6.x86_64                                                                                                                          1/5 
  Installing : libRmath-devel-3.0.1-2.el6.x86_64                                                                                                                    2/5 
  Installing : R-core-3.0.1-2.el6.x86_64                                                                                                                            3/5 
  Installing : R-devel-3.0.1-2.el6.x86_64                                                                                                                           4/5 
  Installing : R-3.0.1-2.el6.x86_64                                                                                                                                 5/5 
  Verifying  : libRmath-devel-3.0.1-2.el6.x86_64                                                                                                                    1/5 
  Verifying  : R-core-3.0.1-2.el6.x86_64                                                                                                                            2/5 
  Verifying  : libRmath-3.0.1-2.el6.x86_64                                                                                                                          3/5 
  Verifying  : R-devel-3.0.1-2.el6.x86_64                                                                                                                           4/5 
  Verifying  : R-3.0.1-2.el6.x86_64                                                                                                                                 5/5 

Installed:
  R.x86_64 0:3.0.1-2.el6                                                                                                                                                

Dependency Installed:
  R-core.x86_64 0:3.0.1-2.el6            R-devel.x86_64 0:3.0.1-2.el6            libRmath.x86_64 0:3.0.1-2.el6            libRmath-devel.x86_64 0:3.0.1-2.el6           

Complete!

[root@oel ~]#

Check that everything works fine.

[root@oel ~]# R

Oracle Distribution of R version 3.0.1  (--) -- "Good Sport"
Copyright (C)  The R Foundation for Statistical Computing
Platform: x86_64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

You are using Oracle's distribution of R. Please contact
Oracle Support for any problems you encounter with this
distribution.

> q();
Save workspace image? [y/n/c]: n
[root@oel ~]# 

Step 2 - Oracle R Enterprise

Download the "server" file from OTN. Unzip the file and run install.sh. Don't forget to set ORACLE_HOME and ORACLE_SID.

[oracle@oel server]$ ./install.sh 
 
Oracle R Enterprise 1.4 Server Installation.
 
Copyright (c) 2012, 2013 Oracle and/or its affiliates. All rights reserved.
 
Checking R ................... Pass
Checking R libraries ......... Pass
Checking ORACLE_HOME ......... Pass
Checking ORACLE_SID .......... Pass
Checking sqlplus ............. Pass
Checking ORACLE instance ..... Pass
Checking ORE ................. Pass
 
Current configuration
  R_HOME               = /usr/lib64/R
  R_LIBS_USER          = /u01/app/oracle/product/11.2.0.4/dbhome_1/R/library
  ORACLE_HOME          = /u01/app/oracle/product/11.2.0.4/dbhome_1
  ORACLE_SID           = orcl
 
Do you wish to install ORE? [yes] 
 
Choosing RQSYS tablespaces
  PERMANENT tablespace to use for RQSYS [SYSAUX]: USERS
  TEMPORARY tablespace to use for RQSYS [TEMP]: 
 
Tablespaces summary
  PERMANENT tablespace = USERS
  TEMPORARY tablespace = TEMP
 
Installing libraries ......... Pass
  Password to use for user RQSYS [ORE1_4RQsys]:  
Installing RQSYS data ........ Pass
Installing RQSYS code ........ Pass
Installing ORE packages ...... Pass
Creating ORE script .......... Pass
Installing ORE migration scripts ...... Pass
 
NOTE: ORE has been enabled for all database users. Next, install the
      supporting packages.
 
      You may create an ORE user with the demo_user.sh script, which
      automatically grants the required privileges. A complete list of
      privileges is available in the script rquser.sql.
 
      To use ORE Embedded R Execution functionality, grant the user
      the RQADMIN role.
 
      Please, consult the documentation for more information.
 
Done
[oracle@oel server]$ 

Step 3 - Install packages

Download the Download the "supporting" file from OTN. Unzip the file and install the packages.

[oracle@oel ~]# R

Oracle Distribution of R version 3.0.1  (--) -- "Good Sport"
Copyright (C)  The R Foundation for Statistical Computing
Platform: x86_64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

You are using Oracle's distribution of R. Please contact
Oracle Support for any problems you encounter with this
distribution.

> install.packages("/home/oracle/supporting/ROracle_1.1-11_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘ROracle’ ...
* DONE (ROracle)
Making 'packages.html' ... done
> install.packages("/home/oracle/supporting/DBI_0.2-7_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘DBI’ ...
* DONE (DBI)
Making 'packages.html' ... done
> install.packages("/home/oracle/supporting/png_0.1-7_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘png’ ...
* DONE (png)
Making 'packages.html' ... done
> install.packages("/home/oracle/supporting/Cairo_1.5-5_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘Cairo’ ...
* DONE (Cairo)
Making 'packages.html' ... done
> 

Step 4 - create a Database User

In this case, I used the demo_user.sh script from the server folder.

[oracle@oel server]$ ./demo_user.sh 
 
Oracle R Enterprise 1.4 Server User Creation.
 
Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved.
 
Checking ORACLE_HOME ......... Pass
Checking ORACLE_SID .......... Pass
Checking sqlplus ............. Pass
Checking ORACLE instance ..... Pass
Checking ORE ................. Pass
 
Current configuration
  ORACLE_HOME          = /u01/app/oracle/product/11.2.0.4/dbhome_1
  ORACLE_SID           = orcl
 
Do you wish to create an ORE user? [yes] 
 
Choosing user
  ORE user to use [rquser]: 
./demo_user.sh: line 215: nawk: command not found
  Password to use for user rquser:  
 
Choosing tablespaces
  PERMANENT tablespace to use [USERS]: 
  TEMPORARY tablespace to use [TEMP]: 
 
Tablespaces and user summary
  PERMANENT tablespace = USERS
  TEMPORARY tablespace = TEMP
  ORE user             = rquser
 
Creating ORE user ............ Pass
 
Done
[oracle@oel server]$ 

Just in case, provide the RQADMIN grant.

[oracle@oel Desktop]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Wed Dec 9 11:17:54 2015

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> GRANT RQADMIN to RQUSER;

Grant succeeded.

SQL> 

Step 5 - Test
Setting up the correct library paths and run demo.

[oracle@oel ~]# R
> .libPaths('/u01/app/oracle/product/11.2.0.4/dbhome_1/R/library');
> .libPaths( c(.libPaths(),'/u01/app/oracle/product/11.2.0.4/dbhome_1/lib')
> library(ORE)
Loading required package: OREbase

Attaching package: ‘OREbase’

The following objects are masked from ‘package:base’:

    cbind, data.frame, eval, interaction, order, paste, pmax, pmin,
    rbind, table

Loading required package: OREembed
Loading required package: OREstats
Loading required package: MASS
Loading required package: OREgraphics
Loading required package: OREeda
Loading required package: OREmodels
Loading required package: OREdm
Loading required package: lattice
Loading required package: OREpredict
Loading required package: ORExml
> 
> ore.connect(user="rquser", sid="orcl", host="localhost", password="oracle", port=1521, all=TRUE);
> ore.is.connected()
[1] TRUE
> ore.ls()
character(0)
> 
> demo("aggregate", package = "ORE")


 demo(aggregate)
 ---- ~~~~~~~~~

Type    to start : return

> #
> #     O R A C L E  R  E N T E R P R I S E  S A M P L E   L I B R A R Y
> #
> #     Name: aggregate.R
> #     Description: Demonstrates aggregations
> #     See also summary.R
> #
> #
> #
> 

Great. Everything works fine. Enjoy.

My first Hadoop application

I am continuing  my trip into Hadoop technologies. Now, it's time to run my first application. I decided to run the classic WordCounts example (you can get the source code here), so I have created the WordCounts.java file. Next, I have compiled the code by using the following code:

hadoop com.sun.tools.javac.Main WordCount.java

The next part is the most difficult - you need to create the jar file. I am not a java guru, so I don't really know what should I get at the end, so I've created the jar using the following command:

jar cf wc.jar WordCount*.class

I will come back to the structure of this wc.jar later, so now Jar file is created and we need only to run it, but before that the hdfs directories should be created:

runas /user:hadoop cmd

%HADOOP_HOME%\bin\hadoop fs -mkdir -p /test_project/in
%HADOOP_HOME%\bin\hadoop fs -put G:\test\gena_test.txt /test_project/in

Everything is ready now, running the application:

hadoop jar G:\test\wc.jar WordCount /test_project/in /test_project/out

Doesn't work!!! Getting the following error:

java.lang.RuntimeException: java.lang.ClassNotFoundException: WordCount$TokenizerMapper

Hm, this is unexpected. Le't google it. At the beginning, I thought that the problem is related with environment variables, therefore I've set all of them (%JAVA_HOME%, %PATH%, %HADOOP_CLASSPATH%) - doesn't help. The next, I've discovered the following warning:

WARN ... No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).

Let's resolve this:

  job.setJar("G:\\test\\wc.jar"); 
//job.setJarByClass(WordCount.class);

But, the error is still shown up. After a couple of hours of useless attempts to resolve this I've found the following topic on stackoverflow.com. The problem was resolved by adding the package into the java class. I tried this and OOOO miracle, the issue is resolved. I think the issue is related with jar file. The jar file I've created previously had two folders :

• test - contains all class files;
• Meta-INF

so I've added the "test" package in the java class, recompiled the class and run it like this:

hadoop jar G:\test\wc.jar test.WordCount /test_project/in /test_project/out

Enjoy.

PS. I've attached the iPhone 6 picture (phone isn't in very good condition I can say :) ). It belongs to one of my associates. The weirdest thing is - it works properly :)

  

Prince2 Certification

The second week of this year I had spend on Prince2 training. I can say, that the training was intense - every day I had to read ~80 pages and learn a lot of stuff. Additionally, the training included two exams (second and fifth days) :). Well, today I have received the result and I am glad to say that I'm a qualified Prince2 Practitioner.

dsSize function and TTSize utility in TimesTen

I didn't have an opportunity to write anything lately, because had spent last two months in Nottingham.  But now I have a couple days to relax, so I have decided to write about one useful function and one utility in TimesTen. 7.0.5 version is used for the demo (I understand that this is a very old version, but I am a lazy person :) )

As you know, TimesTen stores all information into the memory (RAM) and obviously there are limits for RAM space (nowadays you can buy a server with a couple of TBs of RAM, but still it is not 100 TBs). In this situation, TimesTen users should know how to control the space and how to determine the necessary volume of memory for your  system. I have already written one post about the memory allocation here, but now I would like to describe the following:

• dsSize function
• ttSize utility

The first one is used to reports the current sizes of the permanent and temporary database partitions. For example:

[oracle@localhost info]$ ttisql

Copyright (c) 1996-2008, Oracle.  All rights reserved.
Type ? or "help" for help, type "exit" to quit ttIsql.
All commands must end with a semicolon character.


Command> connect dslab1;
Enter password for 'ttdemo': 
Connection successful: DSN=dslab1;UID=ttdemo;DataStore=/app/oracle/product/datastore/dslab1;DatabaseCharacterSet=CL8MSWIN1251;ConnectionCharacterSet=US7ASCII;DRIVER=/app/oracle/product/7.0.5/TimesTen/tt70/lib/libtten.so;LogDir=/app/oracle/product/datastore/dslab1/log;PermSize=120;TempSize=50;TypeMode=0;
(Default setting AutoCommit=1)
Command>
Command> dssize;

  PERM_ALLOCATED_SIZE:      122880
  PERM_IN_USE_SIZE:         1222
  PERM_IN_USE_HIGH_WATER:   1372
  TEMP_ALLOCATED_SIZE:      51200
  TEMP_IN_USE_SIZE:         3587
  TEMP_IN_USE_HIGH_WATER:   3587

Command>

I just created the connection and checked current sizes of the permanent and temporary database partitions. If you add more information in your database, the size will grow. For example, let's create two tables and insert a couple of rows and after that compare results:

[oracle@localhost sql]$ ttisql -f lab1.sql dslab1

Copyright (c) 1996-2008, Oracle.  All rights reserved.
Type ? or "help" for help, type "exit" to quit ttIsql.
All commands must end with a semicolon character.

connect "DSN=dslab1";
Enter password for 'ttdemo': 
Connection successful: DSN=dslab1;UID=ttdemo;DataStore=/app/oracle/product/datastore/dslab1;DatabaseCharacterSet=CL8MSWIN1251;ConnectionCharacterSet=US7ASCII;DRIVER=/app/oracle/product/7.0.5/TimesTen/tt70/lib/libtten.so;LogDir=/app/oracle/product/datastore/dslab1/log;PermSize=120;TempSize=50;TypeMode=0;
(Default setting AutoCommit=1)

run "lab1.sql";

set autocommit 0;

create table emp (id number not null,
                  creation date, 
                  name_1   varchar2(40),
                  name_2   varchar2(40),
                  surname  varchar2(40),
                  dept_id  number,
                  Primary key (id) );

commit;
create table dept (dept_id    number not null,
                   dept_name  varchar2(40), 
                   Primary key (dept_id) );

commit;

insert into dept (dept_id, dept_name) values (1, 'Sales');
1 row inserted.

...

insert into dept (dept_id, dept_name) values (9, 'CEO office');
1 row inserted.

commit;

insert into emp (id, creation, name_1, name_2, surname, dept_id) values (1, sysdate, 'Pety1','Petrovich1', 'Ivanov1',1);
1 row inserted.

…

insert into emp (id, creation, name_1, name_2, surname, dept_id) values (100, sysdate, 'Pety100','Petrovich100', 'Ivanov100',9);
1 row inserted.

commit;

exit;
Disconnecting...
Done.


Command> tables;
   ...
  TTDEMO.DEPT
  TTDEMO.EMP
   ...
Command> dssize;

  PERM_ALLOCATED_SIZE:      122880
  PERM_IN_USE_SIZE:         1372
  PERM_IN_USE_HIGH_WATER:   1372
  TEMP_ALLOCATED_SIZE:      51200
  TEMP_IN_USE_SIZE:         3650
  TEMP_IN_USE_HIGH_WATER:   3713

We see the difference in PERM_IN_USE_SIZE values as expected. ttSize utility is used to estimates the amount of space that a given table in the data store will consume when it grows to include particular amount of rows. For example, we created the table emp, so let's have a look how we can use ttSize utility:

[oracle@localhost ~]$ ttSize -tbl emp -rows 1000000 dslab1
Enter password for 'ttdemo': 

Rows = 1000000

Total in-line row bytes = 207078073

Indexes:
  T-tree index TTDEMO.EMP adds 10541265 bytes
  Total index bytes = 10541265

Total = 217619338

This command shows the approximate amount of memory required for storing 1M rows in emp table.

Hadoop on Windows

A lot of people are interested in Hadoop technology these days and currently I am participating in one project related with this technology. The first task was to install Hadoop - easy, right? But I had been told that we were going to use Windows OS and there is no opportunity to use Linux instead. I conducted a quick research and found the Hortonworks distribution which can work on Windows. BTW, the Hortonworks is the only Hadoop distribution that works in Windows :). As you see, I didn't have much choice, so I downloaded the software and installation guide.

The first surprise is that Windows Server (2008 and 2012) is supported OS only, but I could manage to find the Windows Server 2008 machine. After that you have to install required software (jdk, python, .net and Visual C++ Distributable Package). No problem. Everything is done. The next step is to run the msi file. I run it and specified all parameters I required. Installation started but suddenly McAfee windows appeared and showed the message that one of the Hadoop files contains a virus and just deleted it. Nice. After that very weird things started happens - I started to get errors like "There is a problem with Windows Installer package. A program run as part of setup didn't finish as expected. Contact the vendor". Hm... I got this error even after I stopped all McAfee services. I am not a Windows expert, therefore I didn't even know what to do :)

After one week struggling with Windows Server 2008 and our firm's security policies I could finally install the Hadoop on Windows. The next post will be about the loading data into Hadoop and Hive.

PS. I would like to thank David Goodhand from Hortonworks support team for his help and very fast response (I didn't expect this to be honest :) ).




 

Oracle News

There are a lot of news related with Oracle these days.

The first one is that the founder and CEO of Oracle Larry Ellison decided to leave a CEO job and became a CTO instead. It is kind of a big change in my opinion. Larry has been leading Oracle for the last 37 years and he has archived a great success, therefore now I would expect a new era with Oracle (good or bad will see).

The second one is that OOW conference is started. The first day was about the Cloud, nothing special. Only one announcement was surprising - Oracle Exalytics X4-4 would contain Oracle in-Memory Database! It means that Oracle TimesTen would be replaced by Oracle DB. I think, it is the end of TimesTen! Very sad.

Chapter 4 - Key Terminology

This chapter contains only definitions - the most boring chapter I've ever read. Don't know even what to write. Quick read and test is passed (5 out of 6). Anyway, have to learn these definitions :(
Hope the next chapter will be more interesting.