A site devoted mostly to everything related to Information Technology under the sun - among other things.

Showing posts with label Business Intelligence. Show all posts
Showing posts with label Business Intelligence. Show all posts

Monday, October 14, 2019

Thursday, April 27, 2017

Amazon Echo tells you what to wear

From Grauniad of the UK -

The real question being: why does one need to take 'selfies'?

I want my fridge to tell me what I should eat. My hoover to tell me where the most dust is to be found in my flat. My kettle to choose for me whether I should have tea or coffee.

I want my life run by Amazon.com.

___________________________

https://lnkd.in/eMW3QwE?


Amazon unveils Echo Look, a selfie camera to help you choose what to wear
www.theguardian.com
Hands-free, voice-powered smart selfie camera takes photos and videos on command while its virtual assistant Alexa gives fashion advice

Tuesday, January 19, 2016

The Promise of Action-able Intelligence


Actionable Intelligence for Social Policy (AISP) is an initiative funded by the John D. and Catherine T. MacArthur Foundation, with the purpose of improving the quality of education, and health and human service agencies’ policies and practices through the use of integrated data systems.


A Q&A session with Dennis Culhane and John Fantuzzo, principal investigators may be found here:

http://www.brookings.edu/blogs/up-front/posts/2016/01/19-promise-integrated-data-butler


Thursday, May 7, 2015

Automatic Statistician


 
Reports are generated automatically and have natural language describing patterns in data.

Sunday, October 5, 2014

Tuesday, June 11, 2013

Unique in the Crowd

In the paper "Unique in the Crowd: The Privacy Bounds of Human Mobility," published in Nature's Scientific Reports last year, Yves-Alexandre de Montjoye, César A. Hidalgo, Michel Verleysen, and Vincent D. Blondel of MIT and the Universite Catholique de Louvain, examined a dataset of 15 months of anonymous cell-phone data from 1.5 million people in Belgium (most likely).

There were no names, addresses, or phone numbers in the data, yet they argue that "if individual's patterns are unique enough, outside information can be used to link the data back to an individual." In fact, just four points of observation -- time of the call and the nearest cell-phone tower -- were enough to identify 95 percent of individuals in the data et. 

That is, if one makes four calls from four different places over the course of a 15-month period, one's pattern of movement could be identified out of a population two and a half times the size of Washington, D.C.

If someone is able to cross-reference that with the Twitter feed, say,  he could be able to build a pretty good picture of  who be that person .

The pattern still worked when the researchers "coarsened" their sample by using less specific time observations and by lumping multiple cell-phone towers into one.

Evidently, the way people move through the world and share and communicate information is quite distinctive.

de Montjoye  has stated that: "We use the analogy of the fingerprint, in the 1930s, Edmond Locard, one of the first forensic science pioneers, showed that each fingerprint is unique, and you need 12 points to identify it. So here what we did is we took a large-scale database of mobility traces and basically computed the number of points so that 95 percent of people would be unique in the dataset."

Since phone companies need to keep this kind of data for billing and customer service purposes, it seems inevitable that it would sooner or later be put to questionable use by the security agencies of various governments.
 
The authors have an op-ed in the Christian Science Monitor arguing that consumers should be granted more control over and more information about how much of their data is being stored and for what purpose. 

Their study "shows that when it comes to rich metadata datasets, there are no clear cut between anonymous and not anonymous data. Achieving anonymity is really hard and might even be algorithmically impossible."

The paper may be found here:

http://www.nature.com/srep/2013/130325/srep01376/pdf/srep01376.pdf

Monday, January 21, 2013

MapReduce Book

The book "MapReduce Design Patterns" by Donald Miner and Adam Shook is a good intermediate resource on MapRedue.

Each pattern is explained in context, with pitfalls and caveats clearly identified to help avoid common design mistakes when modeling large data architecture. It also provides a complete overview of MapReduce that explains its origins and implementations, and why design patterns are so important. All code examples are written for Hadoop.  They are:
  • Summarization patterns: get a top-level view by summarizing and grouping data
  • Filtering patterns: view data subsets such as records generated from one user
  • Data organization patterns: reorganize data to work with other systems, or to make MapReduce analysis easier
  • Join patterns: analyze different datasets together to discover interesting relationships
  • Metapatterns: piece together several patterns to solve multi-stage problems, or to perform several analytics in the same job
  • Input and output patterns: customize the way you use Hadoop to load or store data
This book does not have the step-by-step instructions of a "recipe" book, thus avoiding line-by-line breakdowns and delivering a lot of content in its 436 pages. (There is also a usable summary in 30 or so pages.)

Thursday, November 3, 2011

Temporal Databases

Richard T. Snodgrass is a professor at the University of Arizona (http://www.cs.arizona.edu/people/rts/ ) whose whole career is based on temporal databases. You can download a copy of his Morgan-Kaufmann book in (Developing Time-Oriented Database Applications in SQL) in PDF and get the SQL code at the university website.

Wednesday, October 21, 2009

Hadoop

Hadoop is an open-source Java software framework for data-intensive distributed applications. It enables applications to interact with thousands of data nodes.

Hadoop is an implementation of MapReduce which is a programming model for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key.

Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system.

Hadoop was created by Doug Cutting, currently with Cloudera.

Saturday, September 12, 2009

Friday, March 28, 2008

2 Data Visualization Tools

Miner3D has released the version 7.0 of its Miner3D data analytics and visualization software. This release supports interactive & flexible data models and uses the ribbon bar and context menus. One neat feature is the unsupervised visual data clustering - Kohonen's Self-Organizing Maps.

Eye-Sys is a visualization tool for complex data; it can manipulate many different data sources, including text files, 3D models, databases, and streaming data. It also provides an SDK and visual programming environment for the creation of custom scripts.

Saturday, February 16, 2008

Duet

Duet™ is a first-of-its-kind software solution from SAP and Microsoft that enables users to easily and quickly interact with SAP business processes and data via their familiar Microsoft Office environment.

I think this is the correct approach since the installed base of MS Office is estimated to be around 500 million copies world-wide. In particular, I think that the MS Outlook 2007 product should be considered the universal information-management client for today's information workers.

MS Outlook is widely available, it is usually running all the time on the desktop, and using Visual Studio 2008 tools for Office Development one can create new ribbons and panes for the delivery of complex information (including the results of BI & Data mining analysis) to the user's desktop in a convenient and timely manner.

Saturday, September 29, 2007

Insufficiency of Data Models

Malcolm Chisolm has 3 interesting articles on why data modeling alone is not sufficient for the design of databases.

They are:

"Data Models are Not Database Design" in October 2007 issue of DM Review.

"MetaThoughts:The Logical/Physical Challenge in Data Management"

"Is Data Modeling Sufficient for Database Design?"

I think they are worth reading. I specially enjoyed the first one since I had personal experience with reference data and could relate to the validity of his assertions.

BI & Gramm-Leach-Bliley Act

Please be advised that in building BI applications, as soon as you put customer account information into the analysis you trigger the Gramm-Leach-Bliley Act.

About Me

My photo
I had been a senior software developer working for HP and GM. I am interested in intelligent and scientific computing. I am passionate about computers as enablers for human imagination. The contents of this site are not in any way, shape, or form endorsed, approved, or otherwise authorized by HP, its subsidiaries, or its officers and shareholders.

Blog Archive