Creating and managing Secondary user stores via UI with WSO2 API Manager for two Tenants

Get a WSO2 API Manager pack (latest API Manager 1.8.0)
You can download it from here.

Extract the archive file to a preferred location as a folder for API Manager.
I will be referring to it as <APIM_HOME> in this post.

Using MySQL as the database
So that we should copy MySQL JDBC driver to <APIM_HOME>/repository/components/lib directory

Need to point WSO2AM_DB to a MySQL DB
For that you need to change the DB details accordingly in master-datasources.xml which is inside
<APIM_HOME>/repository/conf/datasources

ex :
       <datasource>
            <name>WSO2AM_DB</name>
            <description>The datasource used for API Manager database</description>
            <jndiConfig>
                <name>jdbc/WSO2AM_DB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:h2:repository/database  
                     /WSO2AM_DB;DB_CLOSE_ON_EXIT=FALSE</url>
                    <username>wso2carbon</username>
                    <password>wso2carbon</password>
                    <defaultAutoCommit>false</defaultAutoCommit>
                    <driverClassName>org.h2.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource-->

This is the normal block you get in the API Manager pack. But to point to the MySQL DB the bellow lines need to be changed.
               
                    <url>jdbc:mysql://DBlocationIP:Port/DBname?autoReconnect=true</url>
                    <username>username</username>
                    <password>password</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
            
Create a WSO2AM_DB database
Run the mysql.sql (<APIM_HOME>/dbscripts/apimgt) script which is relevant to this DB to create the relevant table

Similar way create 2 DBs for User stores (ex: US1_DB, US2_DB) and run the sql DB scripts (<APIM_HOME>/dbscripts) relevant to them

Start API Manager using the terminal
sh <APIM_HOME>/bin/wso2server.sh

Or you can go inside the <APIM_HOME>/bin and execute the command.
 sh wso2server.sh 

Or if you did not run the DB scripts you can execute the bellow command. But it is always good to run the relevant DB script parallel to the created DB.
wso2server.sh -Dsetup

Log in to the Management Console using the admin user and click Add New Tenant under Configure menu.
Create the Tenants you need, providing relevant details.

Log in to the Management Console from Tenant 1
Navigate to Configure > User Store Management >  Click Add Secondary User Store
Create a user store manager providing relevant details

  ex: creating a JDBC User Store Manager 
  Domain Name : kala1.com
  Driver Name : com.mysql.jdbc.Driver
  Connection URL : jdbc:mysql://DBlocationIP:Port/US1_DB?autoReconnect=true
  Connection Name : MySQLusername
  Connection Password : MySQLpassword

Follow the same steps for Tenant 2 pointing the created US2_DB

Then followed the steps in [1]

*To create the IDN_OAUTH2_ACCESS_TOKEN_A and IDN_OAUTH2_ACCESS_TOKEN_B tables added blocks same as CREATE TABLE IDN_OAUTH2_ACCESS_TOKEN in to mysql.sql file inside <APIM_HOME>/dbscripts/apimgt

Restart the server
Log in from the created Tenants to the API Publisher and Store
Create/Publish APIs
Subscribe and generate access tokens

Created tokens should store separately in IDN_OAUTH2_ACCESS_TOKEN_A and IDN_OAUTH2_ACCESS_TOKEN_B tables

[1] - https://docs.wso2.com/display/AM180/Saving+Access+Tokens+in+Separate+Tables

Test scaling Logistic regression (ML classification algorithm) by partitioning data by Histogram Introduction

Background

In the modern world having good information can be considered as the life force of business. Machine learning is one of the key techniques which enables to identify  useful data patterns and extract other information from raw data. With the rapid growth of data, ability to handle abundant data using simple machine learning methods is  challenged in terms of performance.

To prevent the drawback large-scale machine learning systems are being introduced. main features of large-scale machine learning systems are;

• Scalability
• Dynamic scheduling
• Parallel execution
• Fault tolerance
• Load balancing

Apache Spark and Apache Mhout are some of the projects which provide large scale machine learning capabilities  

Aim

This is a project to test an alternative method to scale the system by creating a model using partitioned training data using a histogram, and applying logistic regression for each partition. Histogram will be used to partition the data. Elaborating more on the aim,

• Logistic regression will be run using the full data set
• Logistic regression will be run using training data which will be randomly categorized
• Logistic regression will be run on the partitioned training data set using histogram
• The output of all the above will be compared and evaluated against each other

Existing work

Sparks provides a powerful library implementation (MLlib) of some common machine learning functionality.

http://spark.apache.org/docs/0.9.1/mllib-guide.html

Apache Mahout gives;

• Scalable to large data sets
• Scalable to support your business case
• Scalable community

http://mahout.apache.org/

MySQL mostly used commands! :)

MySQL is a popular choice of database for use in web applications.When using MySQL you will come across many commands that you can use. I will be listing some of it, which will be often used.


How to log in to MySQL and run commands from the terminal?

mysql -u newuser -p


Steps to change the password -
mysql -u root
mysql> use mysql;
mysql> update user set password=PASSWORD("NEWPASSWORD") where User='root';
mysql> flush privileges;
mysql> quit

You can exit by using exit or quit commands

There are some alternative ways to log in too..

mysql -unewuser -pnewuser; 
mysql --user=newuser --password=newuser;
mysql --user=newuser -pnewuser;
mysql -unewuser --password=newuser ; 

The password appears in these commands so you can always avoid using them.


How to execute an SQL script file?

mysql> source  path/To/The/SQL/Script/filename.sql

How to remove and create a MySQL DB?

mysql> drop database databaseName;

mysql> create database databaseName;


How to grant permission in MySQL?

GRANT ALL PRIVILEGES ON databaseName.* TO userName@machineIP IDENTIFIED BY 'password';

or

GRANT ALL ON databaseName.* TO userName@'%' IDENTIFIED BY 'password';
'%' means all the relevant IPs


Why do we use flush privileges command?
To tell the server to reload the grant tables!

FLUSH PRIVILEGES;


How to shutdown the server using terminal?

mysqladmin -u root -p shutdown;
You will be asked for the password. After entering the password you can see the MySQL server has been shut down.

How to install MySQL in Mac?!

Do you have any trouble installing MySQL in a Mac laptop? 
Tried installing with Homebrew yet still it is not working? 
Tried to uninstall MyQSL and re-install running the downloaded .dmg, yet it didn't work?

I think I might be able to help you with these kind of questions you have already from this post!

How to uninstall brew installed MySQL stuff completely from the machine? 

Execute the below commands one by one in your terminal.

ps -ax | grep mysql
stop and kill any MySQL processes
brew remove mysql
brew cleanup
sudo rm /usr/local/mysql
sudo rm -rf /usr/local/var/mysql
sudo rm -rf /usr/local/mysql*
sudo rm ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
sudo rm -rf /Library/StartupItems/MySQLCOM
sudo rm -rf /Library/PreferencePanes/My*
launchctl unload -w ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
edit /etc/hostconfig and remove the line MYSQLCOM=-YES-
rm -rf ~/Library/PreferencePanes/My*
sudo rm -rf /Library/Receipts/mysql*
sudo rm -rf /Library/Receipts/MySQL*
sudo rm -rf /private/var/db/receipts/*mysql*

Restart your computer just to ensure any MySQL processes are killed

Just to be more sure try to run MySQL
mysql
This shouldn't work!

Install MySQL in Mac? 

Now you can download MySQL from official MYSQL site

Run the downloaded MySQL .dmg

(Double-click the MySQL installer package and follow the steps)

Once the installation has been completed successfully, you will be shown an Install Succeeded message with a short summary.

To work with MySQL you have to follow a few more steps!

Go to the bash profile.
vi ~/.bash_profile
Insert the below line
export PATH=$PATH:$JAVA_HOME/bin:/usr/local/mysql/bin

Go to System Preferences
Click the MySQL icon
From the pop up window, click Start MySQL Server

Now you are ready to use MySQL in Mac!  :)

SQL query analyzer for Mac?!

Installing the Eclipse SQL Explorer Plugin on Mac

Download and Install Eclipse SQL Explorer Plugin

Open Eclipse
From within Eclipse, choose Help -> Install New Software 
Click Add
Give the URL 'http://eclipsesql.sourceforge.net/' in the 'Work with' field
Fill the pop up dialogue box fields
You should now see 'SQL Explorer' in the list
Click Next
The initializing will happen
A new screen should open which allows you to select 'SQL Explorer'
After selecting it, click Finish
Downloading will start

*******************************************

The SQL Explorer plugin should now be installed and available as a new perspective
You can start using it by selecting it in the Open Perspective list
Window -> Open Perspective -> Other -> SQL Explorer
Once you have opened the SQL Explorer, you will need to add a new connection
In the Connections tab, click the ‘New Connection Profile’ icon or right click anywhere inside the empty list and choose it
This will open the ‘Create New Connection Profile’ window.

Here you won't be able to work unless you add the correct driver!

Download the Driver you want to work with

e.g. If you are trying to run a MSSQL query then you need the Microsoft JDBC Driver

Download the JDBC Driver
You can download it from Microsoft JDBC Driver
After downloading, extract it and open

Open the 'Create New Connection Profile' window in SQL Explorer
Click the New Driver button
On the 'Create New Driver' screen fill in the Name field with something like 'MSSQL JDBC Driver'
In the Example URL field fill in the following:
jdbc:sqlserver://IP-Address:Port;databaseName=DBName
Click the Extra Class Path button
Click Add and then select the 'mssql.jar' file you downloaded
Click the List Drivers button
This should automatically fill in the Driver Class Name field at the bottom with 'com.mssql.jdbc.Driver'

*If not extract the 'sqljdbc4.jar' (when doing this you can copy this to some other location and extract it) and go inside that folder (com/microsoft/sqlserver/jdbc)
This is done to check the Driver name.
Now type 'com.microsoft.sqlserver.jdbc.SQLServerDriver' as the Driver Class Name
Click OK


Now you are ready to run any query you want! :)

More about Homebrew commands to use in Mac!

brew doctor
If you think you need to troubleshoot your Homebrew installation by this the output will tell you what is wrong and generally how to fix it.


brew update 
This updates the versions and availability of packages you could install. You want to run this often to keep your Homebrew installation up to date with the latest available software.

brew upgrade
This will upgrade your installed packages.

brew search [insert search term here]
This command will search the possible packages so you can find the one you want.

brew install [insert package name here]
This will install a software package onto your system.

brew list
This will list all the packages that are currently installed on your system.

brew info [insert package name here]
Sometimes after you install a package, you get a screen full of information or configuration steps. You can get the information on that.

brew uninstall [insert package name here]
When you want to remove a package you previously installed. You can use this!

How to install "stuff" on Mac in an easy way?!

Installing software on a Mac is  different from installing software on Windows. There isn’t just one way to install applications on a Mac, either there are several different ways, depending on the application you want to install.

There is a simple command you can use to install things you need, and that is..

brew install

To use this we have to install Homebrew in Mac. Homebrew is known as a package management system for your Mac :)

To install Homebrew run the following command :
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)

How to use the $ brew install?

To install using brew you have to first make sure that Homebrew is update to date and ready to use.

brew update

 (What this does is update the versions and availability of packages you could install)

brew doctor

 (If you think you need to troubleshoot your Homebrew installation you could use this. The output will tell you what is wrong and generally how to fix it.)

brew upgrade

 (This will upgrade your installed packages.)

 And then you can install using the brew install command.
e.g. brew install mysql

Maven

What is Maven?

Maven is a build automation tool used primarily for Java projects.
So as Maven is a Java tool, you must have Java installed in order to proceed. 
Depending on your network setup, you may require extra configurations too.

To download Apache Maven click on the link

• Extract the distribution archive to the directory you wish to install Maven.
  (These instructions assume you chose /usr/local/apache-maven)
• In a command terminal, add the M2_HOME environment variable.
  e.g. export M2_HOME=/usr/local/apache-maven/apache-maven-3.2.5.
• Add the M2 environment variable.
  e.g. export M2=$M2_HOME/bin.
• Optional: Add the MAVEN_OPTS environment variable to specify JVM properties.
  (This environment variable can be used to supply extra options to Maven)
• Add M2 environment variable to your path
  e.g. export PATH=$M2:$PATH.
• Make sure that JAVA_HOME is set to the location of your JDK.
  e.g. export JAVA_HOME=/usr/java/jdk1.7.0_51
  $JAVA_HOME/bin should be in your PATH environment variable
• Run mvn --version to verify that it is correctly installed.
 

 

For more details you can refer to Apache Maven