Collect your linkedin socail contact on MongoDb

I haven;t written my blog for sometimes.These days I write a simple application to collect my Linked social contact into MongoDB from BlueMix.I will share my experience.I put my code at jazzhub,you can get complete code from here.

LinkedIn  is a business-oriented social networking service.People can know other one's working status and job info from this website.Sometimes poeple want to save his connection into his database for information collection.

When you use my application to deploy on the bluemix,the first things you need to do you need to apply for a key for API usage.You can apply the key from https://developer.linkedin.com/apply.

You can fill the below form,after that you can get the oath key.

The key class is  SocialServlet .

package br.com.ibm.bluemix.mongodb;

import java.io.IOException;

import java.io.PrintWriter;

import java.util.List;

import java.util.Map;

import javax.servlet.Servlet;

import javax.servlet.ServletException;

import javax.servlet.annotation.WebServlet;

import javax.servlet.http.HttpServlet;

import javax.servlet.http.HttpServletRequest;

import javax.servlet.http.HttpServletResponse;

import org.cloudfoundry.runtime.env.CloudEnvironment;

import br.com.bigdata.linkedin.api.ConnectionsAPI;

import br.com.bigdata.linkedin.api.ProfileAPI;

import br.com.bigdata.linkedin.factory.ClientFactory;

import com.google.code.linkedinapi.client.LinkedInApiClient;

import com.google.code.linkedinapi.schema.Person;

import com.mongodb.BasicDBObject;

import com.mongodb.DB;

import com.mongodb.DBCollection;

import com.mongodb.MongoClient;

import com.mongodb.MongoClientURI;

@WebServlet("/Social")

public class SocialServlet extends HttpServlet implements Servlet {

private static final String CONSUMER_KEY_OPTION = "";

private static final String CONSUMER_SECRET_OPTION = "";

private static final String ACCESS_TOKEN_OPTION = "";

private static final String ACCESS_TOKEN_SECRET_OPTION = "";

private static final int QUERY_MAX = 20;

protected void doGet(HttpServletRequest request,

HttpServletResponse response) throws ServletException, IOException {

PrintWriter out = response.getWriter();

try {

String connURL = getServiceURI();

MongoClient mongoClient = new MongoClient(new MongoClientURI(

connURL));

ClientFactory client = new ClientFactory(

SocialServlet.CONSUMER_KEY_OPTION,

SocialServlet.CONSUMER_SECRET_OPTION);

final LinkedInApiClient accessApi = client.getFactory()

.createLinkedInApiClient(SocialServlet.ACCESS_TOKEN_OPTION,

SocialServlet.ACCESS_TOKEN_SECRET_OPTION);

long timeStart = System.currentTimeMillis();

System.out.println("Process start..");

ProfileAPI profileAPI = new ProfileAPI(accessApi);

DB mongoDB = mongoClient.getDB("db");

DBCollection usersProfile = mongoDB.getCollection("usersProfile");

BasicDBObject userDocument = new BasicDBObject();

int countQuery = 0;

Person profile = profileAPI.getProfile();

ConnectionsAPI connection = new ConnectionsAPI(accessApi);

List<Person> peopleConnection = connection.getConnections()

.getPersonList();

String parentId = profile.getId();

while (SocialServlet.QUERY_MAX > countQuery++) {

userDocument.put("lastName", profile.getFirstName());

if (profile.getLocation() != null) {

System.out.println("lastName is"

+ profile.getLocation().getName());

}

System.out.println("lastName is" + profile.getFirstName());

userDocument.put("lastName", profile.getLastName());

System.out.println("lastName is" + profile.getLastName());

userDocument.put("headline", profile.getHeadline());

System.out.println("headline is" + profile.getHeadline());

userDocument.put("summary", profile.getSummary());

System.out.println("summary is" + profile.getSummary());

userDocument.put("profileUrl", profile.getPublicProfileUrl());

System.out.println("profile url"

+ profile.getPublicProfileUrl());

userDocument.put("profileId", profile.getId());

userDocument.put("birthdate", profile.getDateOfBirth());

userDocument.put("profile birthdate", profile.getDateOfBirth());

System.out.println(" birthdate" + profile.getDateOfBirth());

userDocument.put("parentUser",

parentId.equals(profile.getId()) ? "" : parentId);

usersProfile.save(userDocument);

userDocument.clear();

if (peopleConnection.size() > 0

&& peopleConnection.size() > countQuery) {

profile = profileAPI.getProfile(peopleConnection.get(

countQuery).getId());

} else {

countQuery = SocialServlet.QUERY_MAX;

}

}

mongoClient.close();

long timeEnd = System.currentTimeMillis();

out

.println("Your Linked social contact has been saved in the mongodb");

System.out

.println("Time schedule " + (timeEnd - timeStart) + " ms");

} catch (Exception e) {

out.println("Failed: " + e.getMessage());

e.printStackTrace();

}

}

public String getServiceURI() throws Exception {

CloudEnvironment environment = new CloudEnvironment();

if (environment.getServiceDataByLabels("mongodb").size() == 0) {

throw new Exception("No MongoDB service is bund to this app!!");

}

Map credential = (Map) ((Map) environment.getServiceDataByLabels(

"mongodb").get(0)).get("credentials");

System.out.println(" I have get credentials url"+(String) credential.get("url"));

return (String) credential.get("url");

}

}

Notes:You need to add cloudfoundry.jar to get the MongoDB credential url.In order to use Linked API,you need to download the api jar from the https://code.google.com/p/linkedin-j/

After export the war from the eclipse project ,you can use cf command to deploy this way to the BlueMix.

Use command like:

cf p Mysocailcontact -p social.war 

Then bind mongodb service to the Mysocailcontact app.

After execute the below url

http://mysocailcontact.mybluemix.net/Social

You can find the log we wrote in the code on the bluemix.

Login into the Mongo DB console on the bluemix,you can find the record on that db,the record will save my connection from this website.

java based MonogoDB application on BlueMix

NoSQL databases are finding significant and growing industry use in big data and real-time web applications,it provide way to save unstructured data.
Use Cases of ever increasing unstructured data
    Facebook:
        750 million users are active , 1 in 3 Internet users have a Facebook account
        More than 30 billion pieces of content (web links, news stories, blog posts, notes, photo albums, etc.) shared each month.
        Holds 30PB of data for analysis, adds 12 TB of compressed data daily
    Twitter
        200 million users, 200 million daily tweets
        1.6 billion search queries a day
        7 TB data for analysis generated daily

Here is a simple example showing how to save data to mongodb from java web application on BlueMix

1.Get the code from https://hub.jazz.net/project/communitysample/mongodb-java/overview
2. Unzip the code to your driver

3.If you have install maven,use mvn package to generate the war file.

4.Switch the dos command and use below command to push you war to the bluemix.

   cf p javamongodb  -p java-mongodb-sample.war

5.Create your mongodb service
cf create-service mongodb 100 mongodbLinedIN

6.Bind your mongodb service

cf bind-service javamongodb mongodbLinedIN

7.use cf push to make the env effect.

8.Launch the url of the app,you will see the data has been inserted into Mongodb.

Detail code: MainServlet ,we need to add mogodb ,cloudfoundry lib on the classpath

package com.ibm.bluemix.mongodb;

import java.util.*;

import org.cloudfoundry.runtime.env.*;

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;

import com.mongodb.*;

public class MainServlet extends HttpServlet implements Servlet {


  protected void doGet(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
    PrintWriter out = response.getWriter();
    try{
        String connURL = getServiceURI();
        MongoClient mongo = new MongoClient(new MongoClientURI(connURL));

        DB db = mongo.getDB("db");

        DBCollection table = db.getCollection("user");


        BasicDBObject document = new BasicDBObject();
        document.put("name", "Tom");
        document.put("age", 30);
        document.put("createdDate", new Date());
        table.insert(document);

        BasicDBObject searchQuery = new BasicDBObject();
        searchQuery.put("name", "Tom");

        DBCursor cursor = table.find(searchQuery);

        while (cursor.hasNext()) {
            out.println( "Inserted: " + cursor.next());
        }

        BasicDBObject query = new BasicDBObject();
        query.put("name", "Tom");

        BasicDBObject newDocument = new BasicDBObject();
        newDocument.put("name", "Tina");

        BasicDBObject updateObj = new BasicDBObject();
        updateObj.put("$set", newDocument);

        table.update(query, updateObj);

        BasicDBObject searchQuery2 = new BasicDBObject().append("name", "Tina");

        DBCursor cursor2 = table.find(searchQuery2);

        while (cursor2.hasNext()) {
            out.println( "Updated: " + cursor2.next());
        }

        out.println("Success!!");

    } catch (Exception e) {
        out.println("Failed: " + e.getMessage());
        e.printStackTrace();
    }
  }

  public String getServiceURI() throws Exception {
    CloudEnvironment environment = new CloudEnvironment();
    if ( environment.getServiceDataByLabels("mongodb").size() == 0 ) {
        throw new Exception( "No MongoDB service is bund to this app!!" );
    }

    Map credential = (Map)((Map)environment.getServiceDataByLabels("mongodb").get(0)).get( "credentials" );

    return (String)credential.get( "url" );
  }

}

Cloud Foundry vs OpenStack

Cloud Foundry is an open platform as a service, providing a choice of clouds, developer frameworks and application services. Initiated by VMware, with broad industry support, Cloud Foundry makes it faster and easier to build, test, deploy and scale applications. It is an open source project and is available through a variety of private cloud distributions and public cloud instances, including CloudFoundry.com.

OpenStack is a global collaboration of developers and cloud computing technologists producing the ubiquitous open source cloud computing platform for public and private clouds. The project aims to deliver solutions for all types of clouds by being simple to implement, massively scalable, and feature rich. The technology consists of a series of interrelated projects delivering various components for a cloud infrastructure solution.


Cloud Foundry is PAAS,while OpenStack  is IAAS.You can deploy cloud foundry on the openstack.You can watch this video to know the relationships between them.

Learn open source from codeschool

BlueMix platform support lots of open source language,most of developers might master 2-3 language,but if I need to learn some new languages quickly,if there any place for me to learn?

These days I am fascinated with codeschool website.It provides lots of course for you to study and most of them are free.It is an online learning destination for existing and aspiring developers that teaches through entertaining content.

For example if you want to learn git, you can visit this link to use interactive way to study it.It provide the simulate env ,so we don;t need to install anythings on your local side.

The below are part of course provide in this website.Just enjoy it.

Manage MySql,PostgreSQL on the BlueMix

Actually BlueMix don;t provide the solution to operate   db like mysql,mongodb,PostgreSQL etc on the web UI.When people develop their application usually will create db or insert data for these open source db on the BlueMix.So we need a solution to manage these db on the BlueMix.I have already discussed about to maintain mongodb on the BlueMix in the previous post.Today I will talked about how to use manage mysql on the BlueMix.

Deploy phpMyAdmin on BlueMix

phpMyAdmin is a free software tool written in php, intended to handle the administration of MySQl over the Web. phpMyAdmin supports a wide range of operations on MySQL, MariaDB and Drizzle. Frequently used operations (managing databases, tables, columns, relations, indexes, users, permissions, etc) can be performed via the user interface, while you still have the ability to directly execute any SQL statement.

In the BlueMix category,we don;t have php runtime support that.So we will use php buildback to support that.I list the most import steps for you .

1.Since we need to use cf tool to deploy on the BlueMix,you need to download and install it from github.
https://github.com/cloudfoundry/cli

2.We use cf command or web console to create our mysql service.Before create the service,make sure you have use cf login -a api.ng.bluemix.net command to login into it.
create the MySQL service that the application will use. The command syntax is:

cf create-service SERVICE PLAN SERVICE_INSTANCE​

The service name is mysql, the plan is 100, and the service instance name is mysql_NMU. So you'll create your MySQL service like this:

cf create-service mysql 100 mysql_NMU

Here I create mysql service name is called mysql_NMU

3.The next step you need to download or clone the phpMyAdmin package from github from below address
https://github.com/dmikusa-pivotal/cf-ex-phpmyadmin
4.After you donwoload the installation package and open the htdocs\config.inc.php file.
Located the line 28,change 'cleardb-n/a' to 'mysql-5.5' .Below is the code after change.

/*
 * Read MySQL service properties from _ENV['VCAP_SERVICES']
 */
$services = json_decode($_ENV['VCAP_SERVICES'], true);
$service = $services['mysql-5.5'][0]; // pick the first service

4. Since we don;t want use the manifest.yml shipped in the default file,so we can use below command to push on the BlueMix.

cf push phpmyadminbluemix -b https://github.com/dmikusa-pivotal/cf-php-build-pack.git -m 128M --no-manifest --no-start

Bind previous created mysql service
cf bind-service phpmyadminbluemix  mysql_NMU

The we can start our application use below command

 cf start phpmyadminbluemix

5.After start the app,we need to use mysql credential to login into it.You can double click the application,click the runtime menu and find user name and password from Environment Variables.

6.Launch the url from the http://phpmyadminbluemix.mybluemix.net/,you can see below login screenshot.

7.Use the username and password provide by step5,you can login into it and maintain the mysql on the Cloud.

Deploy WebSQL on the BlueMix

WebSQL is the ultimate desktop replacement for managing your MySQL databases over the web. With interface that works just like your favourite desktop applications, you don't need to keep switching over webpages to get simple things done. Just login to your database and manage your database as if your are working on your desktop.It support MySQL, SQLite and PostgreSQL three dbs.

You can download the latest package from http://sourceforge.net/projects/mywebsql/files/stable/mywebsql-3.4.zip/download.

Deploy on the BlueMix for this php based application i same with phpMyAdmin.So I don't want to repeat the same steps here.But one place you might need to change on you side.Open the servers.php in the config foler.Change the code like below.If you want to manage PostgreSQL you can uncomment the line ,and comment mysql serice line for that.
/*
* Read MySQL service properties from _ENV['VCAP_SERVICES']
*/
$services = json_decode($_ENV['VCAP_SERVICES'], true);
$service = $services['mysql-5.5'][0]; // if you choose  the mysql service
       //$service = $services['postgresql-9.1'][0];//if use choose the postgresql-9.1
$host = $service['credentials']['hostname'];
$port=$service['credentials']['port'];

$SERVER_LIST = array(
'MySQL'           => array(
                            'host'     => $host. ':' . $port,
                            'driver'   => 'mysql5'
                        )
/* 'Localhost PostgreSQL'     => array(
                            'host'     =>$host. ':' . $port,
                            'driver'   => 'pgsql'
                       ),
*/
);

If your deployment on BlueMix is successful,you should see below screenshot.

So from below two apps deployment guide ,you can try similar solution with me to support other db on BlueMix.

Send email use Node.js on BlueMix

There are many ways to send eamil use node.js.Today I will introduce three ways to send email.

1.Nodemailer is an easy to use module to send e-mails with Node.JS (using SMTP or sendmail or Amazon SES or even your own method) and is unicode friendly - You can use any characters you like.

The whole development is very easy.
a)Install Nodemailer

> npm install nodemailer

b)Write below code
var nodemailer = require("nodemailer");

var smtpTransport = nodemailer.createTransport("SMTP",{
   service: "Gmail",
   auth: {
       user: "XXX@gmail.com",
       pass: "XXX"
   }
});

var mail = {
    from: "XXX@gmail.com",
    to: "XXX@gmail.com",
    subject: "Send Email Using Node.js",
    text: "Node.js New world for me",
    html: "<b>Node.js New world for me</b>"
}

smtpTransport.sendMail(mail, function(error, response){
    if(error){
        console.log(error);
    }else{
        console.log("Message sent: " + response.message);
    }

    smtpTransport.close();
});

c) In node.js enviroment run nodeemail.js,you will see the email has been sent.

d)You can install it from npm. Add a dependency to your package.json and run npm install
package.json contents:
...
"dependencies": {
   "nodemailer": "0.3.29"
},
...

2. Since first way we need to use smtp and also need authenciation,sometimes we don;t have smtp server provided,so we need to use 'sendmail module to handle it .The code is more easy.

sendmail = require('sendmail')();

sendmail({
    from: 'XXXX@gmail.com',
    to: 'XXX@cn.ibm.com',
    subject: 'test sendmail',
    type: 'text/html',
    content: '<h3>hello</h3><p>Hey guys</p>
  }, function(err, reply) {
    console.log(err && err.stack);
    console.dir(reply);
});
Also you can reference this open source code at github.
https://github.com/guileen/node-sendmail

3.Using SendGrid service,you can reference below link.
https://github.com/sendgrid/sendgrid-nodejs#usage

SoftLayer,competitive ISAS

SoftLayer operates a global cloud infrastructure platform built for Internet scale. Headquartered in Dallas and with an infrastructure spanning data centers in the United States, Asia and Europe, SoftLayer’s unified architecture provides unparalleled performance and control, with a full-featured API andsophisticated automation controlling a flexible platform that seamlessly integrates physical and virtual devices, allowing customers to deploy public cloud instances, bare metal servers and turnkey private clouds in real-time, and a global network for secure, low-latency communications. With 100,000 serversunder management, SoftLayer is the largest privately held infrastructure-as-a-service provider in the world with a portfolio of leading-edge customers from Web startups to global enterprises.It is accquired by IBM in 2013.

Now IBM provide the SoftLayer 30-day free trial for enterprise.You can take chance to have a trial at here.

Most of us are not famliar with  SoftLayer technoledge.What SoftLayer can bring us?Below video the SoftLayer CEO will tell you more details about his company and their business.

What makes SoftLayer different? Softlayer differentiates itself along three main axes – Workload I/O intensity, infrastructure control.

What kind of detail service they provide and what is thier feature? If you have seen the below screenshot you might understand it.Everything starts from the physical server based on the x86 platform.  From this core platform your able to order bare metal servers, private clouds (hyper-visors), virtual servers as well as HPC servers all from the portal and any of these servers can be interconnected with each other.

Also SoftLayer provide a web protal for user to manage his cloud on the site.

Predictable Billing: With SoftLayer you get services on a monthly or hourly contract. No long term commitments or surprise usage fees.
This allows you to order services on demand to meet your current workload requirements.
Many competitors add charges for the resources you use.  For example: compute cycles per second, or bandwidth usage for inbound and outbound.  Our systems come with 5TB (bare metal) and 1TB bandwidth outbound and all private network traffic and public inbound traffic is unmetered.  Resources are not shared so you can use 100% of the resources you purchased without being billed anything extra.
Many gaming companies come to us not only for our performance but because they can budget their IaaS needs much easier even when workloads increase.
Price is inclusive of support as well.  You can contact a 24/7 support personnel via ticket, chat or voice anytime of the day for no extra cost.  Many providers will charge extra or not provide any support unless you are one of their primary accounts.

If  you are interested it,you can start to apply for a trial first,