Monday, July 7, 2014

Data Science


<a href="http://blog.datacamp.com/statistical-language-wars-the-infograph/" >
<img src="http://datacamp.wpengine.com/wp-content/uploads/2014/05/infograph.png" alt="Statistical language wars: SAS vs R vs SPSS" /></a><br/>Source: <a href="http://blog.datacamp.com">blog.datacamp.com</a><br/>

Thursday, March 27, 2014









Internet of Things
Source: BestComputerScienceDegrees.com

“On the Internet, no one knows you’re a toaster.” That might be what people say when the Internet of Things (IoT) — aka, Internet of Everything (IoE) — has widespread use, instead of just the 10 billion “things” that are currently connected online. When that time comes, anything and everything could be connected.
While only a fraction of connected things fall into the familiar categories of web servers, home Wi-Fi routers, smart phones, tablets game consoles, camera, there are a growing number of such things, including sensors for home automation. We have a look at the numbers surrounding the IoT.

The Internet of Things: By the Numbers

Overview: What is the IoT and How it Works

  • 1999 – The term “Internet of Things” coined by Kevin Ashton.
  • Smart planet: The “things” in the “Internet of Things” includes a wide range of things. In addition to what’s mentioned above, the list of potentially connected things are:
  • Home automation sensors (for thermostats, smoke detectors, light bulbs, refrigerators, washing machines, water sprinklers) for smart homes and smart buildings; smart factories; smart cars; smart grids for smart cities.
  • Other devices /things; solar panels, cars, trucks, shipping vessels, first aid kits, windmills, industrial robots. Even insects, cows and other animals have been fitted for wireless sensors that transmit a variety of data — either location or environmental conditions.
  • Sensors range in functionality, including detecting changes in temperature, pressure, humidity, movement and more.
  • Other places we’ll already find smart sensors: contact lenses (GoogleX project), parking spaces, trash systems, and stores — amongst many other things.
  • Not all of IoT things are directly connected to the Internet. Some things, such as sensors, communicate wirelessly to each other (M2M — machine-to-machine) over a localized network, and to some sort of base device which is Internet-connected.
  • Other forms of IoT communication include M2P / P2M (machine-to-person / person-to-machine) and P2P (in this case person-to-person, not peer-to-peer).
  • Wireless communication technologies for non-autonomous things (such as sensors) include RFiD, NFC, Wi-Fi, BLE (Bluetooth Low Energy), XBee, Zigbee, Wireless M-Bus, amongst others.
  • To able to communicate via the Internet requires an IP (Internet Protocol) address to uniquely identify each thing.
  • The older IPv4 (Internet Protocol version 4) system that most computers and older smartphones and tablets use is limited to identifying a maximum of about 4.3 billion addresses.
  • The world is running out of IPv4 addresses, thanks primarily due to the increased use of 4G smartphones and tablets. Some parts of the world have already run out of IPv4 addresses: Europe, the Middle East and parts of Central Asia in 2012; the rest of Asian in 2011. The U.S., Canada and the Caribbean will run out in early Jan 2015.
  • A new system for Internet addresses, IPv6 (version 6) had its World IPv6 Launch Day on Jun 6, 2012, with cellular providers making the largest push.
  • Addresses under IPv6 are much longer, and the maximum number of unique addresses for this system is approximately 3.4×10^38 (34 followed by 37 zeros).
  • Cisco estimates that that is equivalent to about 100 IPv6 addresses for every atom on Earth — meaning we’re unlikely to run out of IPv6 addresses any time soon.

IoT Market Size and Profit Potential

As suggested by numerous reports and white papers by both big players and analysts, the market potential for IoT/ IoE is in the trillions of dollars globally per year and growing. If you want to read in more depth about market forecasts, visit http://postscapes.com/internet-of-things-market-size/.
  • 25 billion autonomous Internet-connected devices by 2015.
  • 31B-50B such devices by 2020, depending on the source.
  • 100B processors shipped, cumulative by 2020, which can process information and communicate the data (via any of several networking technologies).
  • 200B overall connected things (including those that connect to a local M2M network but not directly to the Internet) by 2020.
  • 1.5B vehicles worldwide by 2020, excluding railway vehicles (streetcars, trams, etc.).
  • 3B utility meters by 2020.
  • $4.8 trillion was the global IoT market (technology and services, gross revenue) for 2012.
  • $8.9T is the expected market for 2020 — a CAGR (Compound Annual Growth Rate) of 7.9%.
  • In a Jun 2013 worldwide survey by The Economist of nearly 800 (779) senior business leaders (including 49% C-level execs or board members) representing 19 industries:
    • 96% said their companies would be using IoT in some way within the next 3 years.
    • 76% of companies represented are currently exploring IoT internally (operations, processes) and 74% externally (products, services).
    • 68% said that their companies are already investing budgets in the IoT.
    • 17% are using IoT externally (products, services) or about to do so.
  • $14.4T is Cisco’s conservative estimate of the IoT “Value at Stake” — the net profit potential — globally for the 10 years spanning 2013-2022. (Note: Cisco report uses the term IoE — Internet of Everything.)
  • $1.2T was available for 2013 alone, but approximately $544B was “left on the table,” according to Cisco.
  • Cisco listed 5 factors for enterprise use of IoT that will make up the $14.4T profit potential:
    1. $2.5T — Asset utilization, to reduce costs.
    2. $2.5T — Employee productivity, for labor efficiency.
    3. $2.7T — Supply chain logistics, to eliminate waste.
    4. $3.7T — Customer experience, to increase customer base.
    5. $3.0T — Innovation, to reduce time to market.
  • Cisco also gave what they felt were 8 important real-world use cases representing approximately $6.18T of $14.4T, selected to help business leaders to start getting their company involved in IoE:
    1. $1.95T total value at stake — Smart factories.
    2. $1.95T — Connected marketing and advertising. E.g., through location-based services, viral marketing and mobile advertising.
    3. $757B — Smart grid.
    4. $635B — Connected gaming and entertainment.
    5. $359B — Smart buildings.
    6. $347B — Connected commercial ground vehicles.
    7. $106B — Connected healthcare and patient monitoring.
    8. $78B – Connected private college education.
  • At the time of the Cisco publication (2013), over 99% (99.4%) of devices that might one day be connected are currently not connected.
  • Which means only about 10B “things” of 1.5T globally are currently connected, but that number is growing.
  • Big Data will be an important of managing the IoT, since connected things will generate data for the purposes of analysis.
  • Similarly, Cloud Computing (in the general sense, which includes cloud data storage) will be another important aspect of managing the IoT.

Notable Home Automation Acquisitions: Nest Joins Google

In Jan 2014, Google announced they were purchasing Nest, a maker of home automation sensor devices — in this case, a smart thermostat and smoke detector, both of which can be controlled by an Apple iOS mobile app.
  • $3.2B cash for 3 year-old startup Nest.
  • $150M financing for Nest earlier in Jan 2014, on a valuation of $2B.
  • $80M financing for Nest in Jan 2013, on a valuation of $800M.
  • CEO Tony Fadell and engineering head Matt Rogers will continue to run Nest.
  • Both are former Apple employees — Fadell former SVP of the iPhone and iPod division, and Rogers an engineer.
  • Nest has 2 patent lawsuits filed against them, one by Honeywell (Feb 2012), the other by BRK (Nov 2013).
  • Google was said to be trialling their own smart thermostat in 2013, which might explain their purchase of Nest.
  • Nest claimed only 40-50K smart thermostat units moved monthly as of Jan 2014.
  • With less than 1% of U.S. households (about 1M units) using Nest controllers, Google paid a huge multiple on Nest’s 12-month trailing revenue — estimated as high as 320 times.
  • Possible reasons for the purchase include: gaining access to home automation data to help build citywide smart grids, and for the potential in integrating these sensors with home automation robots. (Google purchased 8 robotics companies in 2013, the last being Boston Dynamics, makers of creature bots such as Little Dog, Big Dog, WildCat and others.
  • Of course, Google had relatively easy access, as Nest was invested in by VC branch Google Ventures. Nest board member Bill Maris was also a partner at Google Ventures.
  • Nest is the 4th Google Ventures-funded company that Google has acquired: Makani Power, Milk, Bufferbox, Nest, in that order.
A number of telecoms and cable companies, including AT&T and Comcast/ Xfinity are working on their own home automation systems. Apple was apparently not in the picture of companies looking to purchase Nest, despite the key Nest executives being former employees.
For Apple, there’s possibly more opportunity in the connected-marketing space, using their iBeacon geofencing technology, which is being rolled out to at least 100 American Eagle Outfitters retail stores in the USA as of Jan 2014. Apple started rolling out iBeacon use to 254 U.S. Apple stores in Dec 2013.

Internet of Things Security: When Refrigerators Go Bad

Where there’s a way, there’s a will? IoT devices have already been used for cyber attacks, to send spam over email. Proofpoint.com published a report in Jan 2014 about a series of cyber attacks.
  • About 100K email messages per cyber attack campaign and 3 campaigns per day were conducted from Dec 23, 2013 through Jan 6, 2014.
  • Over 750K email messages were sent in total in that period.
  • No device sent more than 10 email messages.
  • 450K IP addresses were involved, of which over 100K (over 25%) were IoT devices — or “Thingbots,” as Proofpoint calls them.
  • Other cyber attack devices included game consoles, multi-media centers, televisions and set-top boxes, and one or more refrigerators — all Internet-connected.

IoT: Smart Creatures

Homing devices for animals are nothing new, but the technologies now being used are smaller, more efficient, and have new abilities, including creating a connected network. Current uses of wireless sensors include cows and bees. Not robotic insects but actual honeybees with attached sensors, these two creatures to the potential “thing” list of the Internet of Things.
  • 1/3 of global food crops are pollinated by honeybees.
  • Honeybees are going through “colony collapse disorder” and dying in large numbers annually — by as much 22-36% population loss in some years.
  • 5,000 bees were each fitted with a tiny RFiD chip by Australian agency CSIRO (Commonwealth Scientific and Industrial Research Organization). The intent is that, since bees are habitual in their daily schedule, being able to track their movements may lead to answers about what is disrupting their populations.
  • The chips, which are said not have an impact on bee flight, are about 2.5mm square (or less than 1/10 inch square).
  • 2.5 mm square — the size of the bee chips. This is less than 1/10 inch square.
  • 1mm square — the next gen of these chips, for use on smaller insects.
  • On the other end of the size spectrum are net-connected cows. Dutch startup Sparked has created a wireless sensor that farmers can use to monitor cows.
  • Each cow’s wireless sensor is expected to generate about 200 megabytes of data per year.
  1. http://www.businesswire.com/news/home/20131003005687/en/Internet-Poised-Change-IDC
  2. http://share.cisco.com/internet-of-things.html
  3. http://www.cisco.com/web/about/ac79/docs/innov/IoE_Economy.pdf
  4. http://www.cmswire.com/cms/internet-of-things/the-internet-of-things-its-big-and-its-here-to-stay-023731.php
  5. http://www.ericsson.com/res/docs/whitepapers/wp-50-billions.pdf
  6. http://www.forbes.com/sites/markrogowsky/2014/01/14/5-reasons-nest-sold-to-google/
  7. http://www.forbes.com/sites/patrickmoorhead/2013/11/04/arm-holdings-study-internet-of-things-iot-is-closer-than-you-might-think/
  8. http://www.forbes.com/sites/sarahcohen/2014/01/14/year-of-the-internet-of-things/
  9. http://www.proofpoint.com/about-us/press-releases/01162014.php
  10. http://www.smartplanet.com/blog/bulletin/can-thousands-of-bee-sensors-save-our-food-supply/
  11. http://www.smartplanet.com/blog/bulletin/google-snaps-up-nest-for-32-billion/
  12. http://techcrunch.com/2014/01/14/double-google-all-the-way/
  13. http://techcrunch.com/2014/01/16/shopkick-starts-100-store-ibeacon-trial-for-american-eagle-outfitters-the-biggest-apparel-rollout-yet/
  14. http://en.wikipedia.org/wiki/IPv6_address
  15. http://www.zdnet.com/blog/networking/ipv6-its-the-end-of-the-internet-as-you-know-it-and-i-feel-fine/2283
  16. http://www.zdnet.com/ipv6-gathering-momentum-7000021240/

Saturday, February 22, 2014

Converting JAR file to OSGI Bundle

Hello All, 

You are reading this post, becasue you googled for OSGI bundle which you included in your pom dependency, but unfortunately you did not find OSGI bundle but ended with simple jar file only. So, in this post, I would like to put in short, how to convert JAR file to OSGI bundle. 

1. Create a maven project in eclipse.
  • When creating, you need to be aware of what will go in artifact Id and group id.
  • Use same groupId and change artifact Id to -osgi. offcourse, you can follow any naming convention, but never give same name as dependency you are adding.(Giving the same name as dependent bundle will not export packages and treats output of this project(osgi bundle) and dependent bundle(jar) as same). Be carefull here.
2. Add "maven-bundle-plugin" plugin to plugins section as shown below.




  org.apache.felix
maven-bundle-plugin 1.2.0 true ${project.artifactId} ${project.groupId}.${project.artifactId} OSGI version of ${project.name} ${export.packages} true 3. Add your dependecy of jar which you want to convert to osgi.


Example:

 
  net.spy
spymemcached 2.10.0

4. Build project with "mvn clean install" from command prompt.

5. Varify the "Export-Package:" section of MANIFEST.MF file, which should list all the packages which are there in your(added dependency) jar file.

6. New bundle is ready to deploy to OSGI container like sling(Felix console).

In above step 2, as we are specifying 
true
, which unpacks all classes under dependent bundles, and "maven-bundle-plugin" plugin is responsible to create osgi bundle including these dependent classes.

You can optionaly exclude some of the packages and have some custom classes and include those classes to this bundle as well by having and  tags under  tags under of this bundle. One more last thing, you can also combine multiple jars by adding maven dependecy to pom.xml file and build single OSGI bundle. This will help to reduce the pain of deploying individual bundles to felix console. Offcourse you have to build this bundle agin if you would like to change any of the dependent jars bundle version. Hope this helps and happy OSGIfying...:-)

References: http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html

Tuesday, February 18, 2014



         Cloud Forensics ---Retrieving Virtual Disks for Forensic Investigation

1. Openstack Installation :
                             The Following are the various ways to install Openstack Cloud Orchestration System

1.Devstack Multi Node Installation :-
       
        We need to have a fresh install of linux on all linux nodes at least on 3 systems so that we can run openstack service’s on different nodes.

Devstack Refers the following Website for Minimal Ubuntu 12.04  Download on all Nodes

2. Opscode Chef Server :-
                              


Summary Of
                                        Design and Implementation of FROST
                    Digital Forensic Tools for the OpenStack Cloud Computing Platform
                                        Josiah Dykstra and Alan T.Sherman



            The Objective of the paper is add forensic tools for the Openstack Cloud platform which operates at the management plane. These Forensic capabilities allows the customers,forensic examiners and law enforcement to acquire trustworthy forensic acquisition of virtual disks,API Logs and guest firewall logs.

                               FROST works at the cloud management plane rather than interacting with the operating system inside the guest virtual machine, thereby requiring no trust in the guest virtual machines.It overcomes non-trivial challenges of remote evidence integrity by storing log data in hash trees and returning evidence with cryptographic hashes.

           The Following assumptions are taken into consideration
                  
                  1. The User driven forensic capabilities are applicable in situations where a cooperative cloud customer is involved in the investigation. That is ,if a malicious customer uses the cloud to commit a crime, the cloud provider will still be required to assist law enforcement in the investigation.

                 2. The Frost tools assume trust in the cloud provider and cloud infrastructure. otherwise, there is a chance of modifying evidence at the provider side so we require trust in the host operating system, hardware and provider.

The paper has contributed following capabilities to FROST:-

1. Implementation of user driven forensic acquisition of virtual disks, API logs and firewall logs from the management plane of openstack.

2. An algorithm for storing and retrieving log data with integrity in a hash tree that logically segregates the data of each cloud user in his or her own subtree.

3. Evaluation results showing that the proposed solution satisfies technological and legal requirements for a acceptance in court .


                        


Specifications and Capabilities :-
                 
                        FROST has three primary components

1. A Cloud user can retrieve image of the virtual disks associated with any of the user’s virtual machines,and validate the integrity of those images with cryptographic checksums.

2.A cloud user can retrieve the logs of all API requests made to the cloud provider using his or her credentials and validate the integrity of those logs.

3.A cloud user can retrieve the Openstack firewall logs for any of the user’s virtual machines,
and validate the integrity of those logs.

A  Scenario which shows the advantage of FROST :-

          An arbitrary cloud customer alice who wants to investigate suspiciously high bandwidth usage from her cloud hosted web server. Aside from the logging of web requests that she does
inside of her own VM.Alice would have a more complete picture of activity if she could also get a record of management activity and meta data about her VMs . The FROST Collects and provides trustworthy API logs, guest firewall logs and virtual disks. These data can help construct a timeline activity and understand an incident.      


Conclusion :-
                                     
        I conclude that the FROST implements the acquisition phase of the forensic process and there are other phases need to be added according to the cloud computing platform .I wanted to re-implement this paper to my best so that I can get good exposure to Cloud platform and ability to add  additional modules required to the FROST .
                   
           
References :-
1. Amazon Web Services: Overview of Security Processes. Available at
http://awsmedia.s3.amazonaws.com/pdf/AWSSecurityWhitepaper.pdf;2011. [accessed 10.28.2012].

2.Clarke, D.E.. Towards Constant Bandwidth Overhead Integrity Checking of Untrusted Data. Ph.D. thesis; MIT; 2005.

3.Crosby, S.A.. Efficient Tamper-Evident Data Structures for Untrusted Servers. Ph.D. thesis; Rice University; 2009.

4.Dykstra,J.,Riehl, D.. Forensic Collection of Electronic Evidence from Infrastructure-As-A-Service Cloud Computing. Richmond Journal of Law and Technology 2012;19. Available at
http://jolt.richmond.edu/wordpress/?p=463.

5.Dykstra, J., Sherman, A.T.. Understanding Issues in Cloud Forensics: Two Hypothetical Case Studies. In: Proceedings of the 2011 ADFSL Conference on Digital Forensics Security and Law. ASDFL; 2011a. p. 191–206.

6.Garfinkel,S..Digital forensics xml and the df xml toolset. Digital Investigation 2012;8(3–4):161–174.

7.Taylor, M., Haggerty, J., Gresty, D., Lamb, D.. Forensic investigation of cloud computing systems. Network Security 2011;(3):4–10.

8.Scientific Working Group on Digital Evidence (SWGDE),.Data Integrity Within Computer Forensics.
Available at :-https://www.swgde.org/documents/Current%20Documents/2006-
04-12%20SWGDE%20Data%20Integrity%20Within%20Computer%20Foren
2006. [accessed 9.16.2012].

9.Ruan, K., Carthy, J., Kechadi, T., Crosbie, M.. Cloud forensics: An overview. In: Advances in Digital Forensics VII. 2011. .

10.National Institute of Standards and Technology,.Digital Data Acquisition Tool Specification.Available at http://www.cftt.nist.gov/Pub-Draft-1-DDA-Require.pdf; 2004. [accessed 9.16.2012].

11.Marty, R.. Cloud application logging for forensics. In: Proceedings of the 2011 ACM Symposium on Applied Computing. New York, NY, USA: ACM; SAC ’11; 2011. p. 178–184.

12.Liand, J., Krohn, M., Mazi`res, D., Shasha, D.. Secure Untrusted Data Repository (SUNDR). .

13.Kundu, A.. Data in the Cloud: Authentication without Leaking. Ph.D. thesis; Purdue University; 2010.























Monday, January 20, 2014

How to create your own HTTP server
After reading this you can create your own http server in your
system. A web server is software installed on a computer that allows
other computers to access specific files on that computer/server. There
are many reasons to create your own home servers. For example: file
sharing, so you can download files from your home computer from
anywhere or you can create a web site on own server etc. Simply said
It works like this; You choose a directory on your computer , in that
directory add folders, files like music,video and etc. When you put the
IP address of your computer the in web browser you can see all the
files from that folder and you can download those files. Let’s now
create a server(HTTP server!) using Apache(a server client):
Requirements:
Broadband internet connection always on
Windows on your computer
Installing Apache Http Server and Configuring :
Download the apache_2.2.10-win32-x86-no_ssl.msi
Start to install by following the steps:
1.Set parameters( for localhost type something like a myserver.com
(doesn’t really matter), also type your email address in field
“Administrator@ Email Address” ) choose where you
want to install it.
2.When you install Apache , go to directory where you installed it (p.e.
“C:\Program Files\Apache Software Foundation\Apache2.2\conf”) ,
here you will find a httpd file.
Open that file with notepad.
After this will appear notepad with long and complicated code, don´t
worry, you must change just 3 things.
3.In notepad file find #DocumentRoot “C:/Program Files/Apache
Group/Apache2/htdocs” and replace with #DocumentRoot “E:\my
server”. Also find #
Apache2/htdocs” and replace with
server is folder where you put files which will appear on your server. In
this example I created that folder on local disc E:. You can create your
folder in any other place, but then type that path here. Find
#AllowOverride None and change to AllowOverride All .
After this, save file like httpd.conf.
4.Create Folder in E drive as "myserver"
5. Type in web addresses http://localhost/ or your IP Address
6. If you want access own server from other computers. You must
forward a port in the router we’re using. The port we need to forward
is port number 80. Why? Because by default it’s the port used for
HTTP. Port forwarding actually means opening a tunnel through the
router so that the router wouldn’t reject the connections that are
trying to connect to it. How to port-forward? With every router it’s
different. You must
also turn off you firewall.
Note: Creating home server is risky,when you open port, there is a
possibility to have someone a breach in your computer .Before you
start, make sure your computer has all the latest patches and
security updates, and that you’ve done a thorough spyware and
virus scan. This tutorial is only for advanced users...