A well-respected online journal is the Virtual Journal of Quantum Information @ www.vjquantuminfo.org/quantuminfo.
It provides links to papers appearing in many specialized disciplinary journals (Low Temperature Physics, Optics Express, Science, Physical Review A, and so on), organized by themes relating to quantum processing such as “information theory” and “algorithms and computation.”
Friday, November 30, 2007
Sunday, November 25, 2007
Jupiter and its moon Io
This NASA montage of images of Jupiter and its volcanic moon Io, was taken by the New Horizons spacecraft's flyby in early 2007.
The Jupiter image is an infrared color composite taken by the spacecraft's near-infrared imaging spectrometer on Feb. 28, 2007.
The Io image, taken on March 1, 2007, is a nearly true-color composite. The image shows a major eruption in progress on Io's night side, at the northern volcano Tvashtar.
Saturday, November 24, 2007
More Graduates Than Jobs
This study purports to show that US colleges are producing more graduates in sciences and mathematics than there are jobs for them.
http://www.urban.org/UploadedPDF/411562_Salzman_Science.pdf
I tend to agree with the conclusions of this study since I recall a shortage of scientists and engineers claim made almost 20 years ago - it led to the creation of an army of unemployable Ph.D.s in all branches of science.
http://www.urban.org/UploadedPDF/411562_Salzman_Science.pdf
I tend to agree with the conclusions of this study since I recall a shortage of scientists and engineers claim made almost 20 years ago - it led to the creation of an army of unemployable Ph.D.s in all branches of science.
This is Innovation
Windbelt, a cheap generator alternative:
http://www.popularmechanics.com/technology/industry/4224763.html?series=37
I found this to be a neat and innovative design; I really liked it!
http://www.popularmechanics.com/technology/industry/4224763.html?series=37
I found this to be a neat and innovative design; I really liked it!
Thursday, November 22, 2007
Robotic Bugs Mingle With Cockroaches
Tiny robots programmed to act like roaches were able to blend into cockroach society, according to researchers studying the collective behavior of insects. Cockroaches tend to self-organize into leaderless groups, seeming to reach consensus on where to rest together. For example, when provided two similar shelters, most of the group tended to gather under the same one.
Hoping to learn more about this behavior, researchers led by Jose Halloy at the Free University of Brussels, Belgium, designed small robots programmed to act like a cockroach.
The robots didn't look like the insects and at first the roaches fled from them, but after the scientists coated the robots with pheromones that made them smell like roaches the machines were accepted into the group, nesting together with the insects.
Given a choice, roaches generally prefer a darker place and the robots were programmed to do the same. When given a choice of a darker or lighter shelter, 75 percent of the cockroaches and 85 percent of the robots gathered under the darker one.
Hoping to learn more about this behavior, researchers led by Jose Halloy at the Free University of Brussels, Belgium, designed small robots programmed to act like a cockroach.
The robots didn't look like the insects and at first the roaches fled from them, but after the scientists coated the robots with pheromones that made them smell like roaches the machines were accepted into the group, nesting together with the insects.
Given a choice, roaches generally prefer a darker place and the robots were programmed to do the same. When given a choice of a darker or lighter shelter, 75 percent of the cockroaches and 85 percent of the robots gathered under the darker one.
Wednesday, November 21, 2007
Tuesday, November 20, 2007
Elastic Compute Cloud
In August 2006, Amazon introduced the EC2 (Elastic Compute Cloud).
Creating virtual machines somewhere across the Internet is what Amazon’s EC2 does. Amazon charges 10 cents for each hour a virtual machine is running. I can cancel or pause at anytime. Coupled with their storage network, it is possible to use precisely the amount of CPU and disk that I need with a remarkably simple setup process. Two minutes after I request my first machine, I’m SSHed into a root shell. My first hour of kicking the tires cost 11 cents. This freedom to experiment makes an enormous difference.
The machine is a virtual instance, with 1.7 Ghz x86 processor, 1.75 GB RAM, 160 GB local disk, and 250 Mb/s of network bandwidth. This costs $0.10/machine/hour. Machines with 4x or 8x specs are available for $0.40 and $0.80.
Persistent storage is independent of the virtual machines for $0.15/GB/month. Transfers within the cloud are free, with data moving between the cloud and the internet costing $0.10/GB on the way in and $0.18/GB for each transferred out. With larger volumes, outbound transfers fall to $0.13/GB.
Creating virtual machines somewhere across the Internet is what Amazon’s EC2 does. Amazon charges 10 cents for each hour a virtual machine is running. I can cancel or pause at anytime. Coupled with their storage network, it is possible to use precisely the amount of CPU and disk that I need with a remarkably simple setup process. Two minutes after I request my first machine, I’m SSHed into a root shell. My first hour of kicking the tires cost 11 cents. This freedom to experiment makes an enormous difference.
The machine is a virtual instance, with 1.7 Ghz x86 processor, 1.75 GB RAM, 160 GB local disk, and 250 Mb/s of network bandwidth. This costs $0.10/machine/hour. Machines with 4x or 8x specs are available for $0.40 and $0.80.
Persistent storage is independent of the virtual machines for $0.15/GB/month. Transfers within the cloud are free, with data moving between the cloud and the internet costing $0.10/GB on the way in and $0.18/GB for each transferred out. With larger volumes, outbound transfers fall to $0.13/GB.
On Global Warming
I am including this topic since so much of the debate rests on rather dubious conclusions drawn from computer models that have not correctly formulated the relevant physical and chemical processes in the atmosphere (say the effect of the clouds on the Earth's radiation budget).
But people seem to be impressed by the size of the computers that produce these numerical results.
I am including Richard Lindzen's 2002 article; it makes eminent sense to me @ http://eaps.mit.edu/faculty/lindzen/153_Regulation.pdf
And some introductory material:
http://www.nationalcenter.org/NPA420.pdf
http://eteam.ncpa.org/files/GlobalWarmingPrimer_low.pdf
And more technical discussions:
http://minerva.simons-rock.edu/~geshel/Huxley/huxleyClimate.pdf
http://home.uchicago.edu/~schmidt/0308061.pdf
There is also the book "Cool it: the Skeptical Environmentalist's Guide to Global Warming" by
Bjorn Lomborg of "The Skeptical Environmentalist" fame & subsequent controversy.
But people seem to be impressed by the size of the computers that produce these numerical results.
I am including Richard Lindzen's 2002 article; it makes eminent sense to me @ http://eaps.mit.edu/faculty/lindzen/153_Regulation.pdf
And some introductory material:
http://www.nationalcenter.org/NPA420.pdf
http://eteam.ncpa.org/files/GlobalWarmingPrimer_low.pdf
And more technical discussions:
http://minerva.simons-rock.edu/~geshel/Huxley/huxleyClimate.pdf
http://home.uchicago.edu/~schmidt/0308061.pdf
There is also the book "Cool it: the Skeptical Environmentalist's Guide to Global Warming" by
Bjorn Lomborg of "The Skeptical Environmentalist" fame & subsequent controversy.
Sunday, November 18, 2007
Transport
Click on the images to see a larger view.
I note here how an original design is being pushed beyond its original conception all over the world; kind of like what happens to software.
I note here how an original design is being pushed beyond its original conception all over the world; kind of like what happens to software.
Saturday, November 17, 2007
4 Easy Steps to SOA
1. One begins with the notion of information models as they exist within the problem domain or enterprise, and defines them to the level of detail where one has a complete understanding of the information, semantics, structure, policies and other attributes that enables one to define the information model.
2. Next, one defines the abstract model, or how to view the information in the context of the SOA. This means taking the physical representations and structures, and turning them into an information layer that makes better logical sense within SOA. This approach and its enabling technology has clear advantages when considering agility, since the visible structures for use within the services are configurable.
3. Next, one defines the services, or behaviors, and how the underlying information models that- just created - are bound to those services. Typically, one creates two types of services—data services and transactional services—but one can easily extend them. Services can be bound into processes and composites as well later.
4. Finally, one defines how all of the services below exist within a process or orchestration model, or a place where business processes are defined and redefined, thus providing the core benefit of SOA—agility.
5. The core idea here is to place volatility into a single domain, or a place where the use of services (and thus the information) can be defined, and redefined, according to the requirements of the business.
There are also ancillary needs to consider:
2. Next, one defines the abstract model, or how to view the information in the context of the SOA. This means taking the physical representations and structures, and turning them into an information layer that makes better logical sense within SOA. This approach and its enabling technology has clear advantages when considering agility, since the visible structures for use within the services are configurable.
3. Next, one defines the services, or behaviors, and how the underlying information models that- just created - are bound to those services. Typically, one creates two types of services—data services and transactional services—but one can easily extend them. Services can be bound into processes and composites as well later.
4. Finally, one defines how all of the services below exist within a process or orchestration model, or a place where business processes are defined and redefined, thus providing the core benefit of SOA—agility.
5. The core idea here is to place volatility into a single domain, or a place where the use of services (and thus the information) can be defined, and redefined, according to the requirements of the business.
There are also ancillary needs to consider:
- A monitoring & event management layer that exists above the business processes to keep an eye on the architecture during operations,
- A governance model,
- A security model,
- The ability to create & enforce policies around the use of the services,
- The ability to bind security down to core granular components, typically through a federated security model.
Why LINQ Is Important
LINQ is a language feature that allows relational operators to be applied to objects. The obvious use for this is to replace embedded SQL, which is enabled by “LINQ to SQL” (often called DLinq). The biggest win: there is no longer the need for the manual mapping of table names, column names and SQL types to runtime classes, fields and CLR types which has always been extremely tedious and error-prone.
Dlinq requires an explicit mapping, a subtype of DataContext, between the SQL and object world views. Generally, this mapping is created by dropping a SQL model onto a design surface and clicking “OK” a few times. Customization is very easy, though but there are many complexities to object-relational mapping and there will be issues here.
Although the syntax of LINQ is determined by the language designer, not the runtime, in the case of both C# and VB and there are keywords such as select, from, and where that will be familiar to any SQL user.
LINQ additionally supports all the common operators of the relational algebra. This means that, just as a SQL query can result in a table with columns that are mixed-and-matched from the source table, so too can you create objects that are mixed-and-matched from their sources! The implications of that are fairly extensive and lead to concepts such as anonymous types and closures that may be unfamiliar to some, but which are pretty easy to understand in their workaday usage.
LINQ is not an attempt to create a unified data model rather a unified query/manipulation facility. In contrast to DLinq, XLinq 1.0 does not revolve around a strongly typed mapping between the object world and that of XML; a tool for such mapping, called LINQ to XSD, is in development but will not be part of the initial release. (Find out more about LINQ to XSD at blogs.msdn.com/xmlteam/default.aspx.)
Two additional fascinating LINQ technologies are extensions to automatically parallelize LINQ (PLinq) and the use of LINQ and reflection APIs (see tinyurl.com/225pd2). Further, the Dynamic Language Runtime Group at Microsoft (those working on the Iron* languages) point to deep equivalences between LINQ expressions and what they are using to generate code at runtime.
The 1.0 release of LINQ is absolutely something that every CLR developer ought to become familiar with.
Dlinq requires an explicit mapping, a subtype of DataContext, between the SQL and object world views. Generally, this mapping is created by dropping a SQL model onto a design surface and clicking “OK” a few times. Customization is very easy, though but there are many complexities to object-relational mapping and there will be issues here.
Although the syntax of LINQ is determined by the language designer, not the runtime, in the case of both C# and VB and there are keywords such as select, from, and where that will be familiar to any SQL user.
LINQ additionally supports all the common operators of the relational algebra. This means that, just as a SQL query can result in a table with columns that are mixed-and-matched from the source table, so too can you create objects that are mixed-and-matched from their sources! The implications of that are fairly extensive and lead to concepts such as anonymous types and closures that may be unfamiliar to some, but which are pretty easy to understand in their workaday usage.
LINQ is not an attempt to create a unified data model rather a unified query/manipulation facility. In contrast to DLinq, XLinq 1.0 does not revolve around a strongly typed mapping between the object world and that of XML; a tool for such mapping, called LINQ to XSD, is in development but will not be part of the initial release. (Find out more about LINQ to XSD at blogs.msdn.com/xmlteam/default.aspx.)
Two additional fascinating LINQ technologies are extensions to automatically parallelize LINQ (PLinq) and the use of LINQ and reflection APIs (see tinyurl.com/225pd2). Further, the Dynamic Language Runtime Group at Microsoft (those working on the Iron* languages) point to deep equivalences between LINQ expressions and what they are using to generate code at runtime.
The 1.0 release of LINQ is absolutely something that every CLR developer ought to become familiar with.
Friday, November 16, 2007
Free Online Applications
Desktoptwo (http://www.desktoptwo.com/) is a Flash-based online desktop with no local installation.
Meebo (http://www.mwwbo.com/) is a Browser-based Web messenger.
Mindomo (http://www.mindomo.com/) is an online mind-mapping tool.
ThinkFree Online (http://www.thinkfree.com/) is a free online productivity suite of 3 applications like MS Word, Excel, and PowerPoint.
Tubes (http://www.tubesnow.com/) supplies shared location space for exchanging documents and files.
Menutopia (http://www.menutopia.com/) is a community driven restaurant guide.
Mvelopes (http://www.mvelopes.com/) is an online personal finance program.
DailyLit (http://www.dailylit.com/) emails small chunks of books to its subscribers based on their preferences.
Cheap Supercomputing
Professor Joel Adams and former student Tim Brom have built a high-performance computer that sits on a desktop and cost only $2,470 to construct. It now could be built for just $1,255.
as Micorbeowulf, it performs at 26.25 gigaflops and measures just 11 x 12 x 17 inches and it does offer the best price-performance ratio—$47.80 per Gflops based on current construction costs. Compare this to Stanford University’s Bio-X supercomputer cluster, for which the price-performance ratio is $109 per Gflops, considered very efficient.
Two technology improvements made the project possible: multicore CPUs and Gigabit Ethernet. Microwulf has four dual-core chips, which communicate via Gigabit Ethernet. Adams has ported 12 applications and his students have written two demonstration programs to show Microwulf’s capabilities.
The construction of the Microwulf makes high performance computing within the reach of small companies. This means that data mining, certain classes of AI applications, and business intelligence application are within reach of small (smaller that 200 staff) organizations.
There are more technical details @ http://www.calvin.edu/~adams/research/microwulf/
as Micorbeowulf, it performs at 26.25 gigaflops and measures just 11 x 12 x 17 inches and it does offer the best price-performance ratio—$47.80 per Gflops based on current construction costs. Compare this to Stanford University’s Bio-X supercomputer cluster, for which the price-performance ratio is $109 per Gflops, considered very efficient.
Two technology improvements made the project possible: multicore CPUs and Gigabit Ethernet. Microwulf has four dual-core chips, which communicate via Gigabit Ethernet. Adams has ported 12 applications and his students have written two demonstration programs to show Microwulf’s capabilities.
The construction of the Microwulf makes high performance computing within the reach of small companies. This means that data mining, certain classes of AI applications, and business intelligence application are within reach of small (smaller that 200 staff) organizations.
There are more technical details @ http://www.calvin.edu/~adams/research/microwulf/
Design
Human artifacts do not have to be ugly. Here is an example of design in alarm clocks; some thing that has been around for more than a 100 years.
http://www.pcmag.com/slideshow/0,1206,l=218600&s=25301&a=218612,00.asp
http://www.pcmag.com/slideshow/0,1206,l=218600&s=25301&a=218612,00.asp
Tech Toys
Luxurious Tech Toys:
http://www.pcmag.com/article2/0,2704,2217823,00.asp
And not so luxurious:
http://www.pcmag.com/article2/0,2704,2217004,00.asp
http://www.pcmag.com/article2/0,2704,2217823,00.asp
And not so luxurious:
http://www.pcmag.com/article2/0,2704,2217004,00.asp
Automated Killers and the Computing Profession
From the November 2007 issue of the IEEE Computer Magazine:
When will we realize that our artificial-intelligence and autonomous-robotics research projects have been harnessed to manufacture killing machines? This is not terminator-style science fiction but grim reality: South Korea and Israel have both deployed armed robot border guards, while other nations—including China, India, Russia, Singa- pore, and the UK—increasingly use military robots.
When will we realize that our artificial-intelligence and autonomous-robotics research projects have been harnessed to manufacture killing machines? This is not terminator-style science fiction but grim reality: South Korea and Israel have both deployed armed robot border guards, while other nations—including China, India, Russia, Singa- pore, and the UK—increasingly use military robots.
Thursday, November 15, 2007
Molecular Computing
MOE is a software system designed by the Chemical Computing Group to support Cheminformatics, Molecular Modelling, Bioinformatics, Virtual Screening, Structure-based-design and can be used to build new applications based on SVL (Scientific Vector Language).
Learn more about it @ http://macresearch.org/review_moe_molecular_operating_environment
Learn more about it @ http://macresearch.org/review_moe_molecular_operating_environment
UnifiedLogic
Check out this neat approach to using FPGAs in embedded design:
http://www.pldesignline.com/howto/showArticle.jhtml;jsessionid=CVUBBOSHYJRNUQSNDLPCKHSCJUNN2JVN?articleID=203100209
And
http://www.eridon.com/?AspxAutoDetectCookieSupport=1
http://www.pldesignline.com/howto/showArticle.jhtml;jsessionid=CVUBBOSHYJRNUQSNDLPCKHSCJUNN2JVN?articleID=203100209
And
http://www.eridon.com/?AspxAutoDetectCookieSupport=1
Wednesday, November 14, 2007
Some groovy stuff: ANTLR
This is a tutorial on how to use ANTLR to build a domain specific language.
http://www.codeproject.com/csharp/sota_expression_evaluator.asp
http://www.codeproject.com/csharp/sota_expression_evaluator.asp
Tuesday, November 13, 2007
Sunday, November 11, 2007
Sentinel Visualizer
Check out this new data exploration and visualization tool ($ 1900 per single user license) @ http://www.fmsasg.com/Products/SentinelVisualizer/.
Free Web Testing Tools
These are 2 tools for functional testing of Web applications. JMeter can generate complex test plans, tracks and report using a large range of options, and validate results via a range of assertion types. This is well designed tool for load testing and performance analysis.
Selenium is a Plug-in which runs directly in a browser. It runs in Internet Explorer, Mozilla and Firefox on Windows, Linux, and Macintosh. Selenium is useful for Browser compatibility testing. It is also useful for system functional testing by creating regression tests to verify application functionality.
There is a nice article in the November 2007 issue of the Software Test & Performance Magazine by Alan Berg (@ http://www.stpmag.com/issues/stp-2007-11.pdf) that discusses these 2 tools in much more detail.
Check them out!
Selenium is a Plug-in which runs directly in a browser. It runs in Internet Explorer, Mozilla and Firefox on Windows, Linux, and Macintosh. Selenium is useful for Browser compatibility testing. It is also useful for system functional testing by creating regression tests to verify application functionality.
There is a nice article in the November 2007 issue of the Software Test & Performance Magazine by Alan Berg (@ http://www.stpmag.com/issues/stp-2007-11.pdf) that discusses these 2 tools in much more detail.
Check them out!
Saturday, November 10, 2007
Outsourcers as Web Platforms
Outsourcing companies such as IBM, HP, Accenture, EDS, and others like them have very extensive and robust IT infrastructures for supporting their necessarily large clients. Their holdings include voice/data networks, databases, servers, telephony devices, storage devices etc.
Some of this infrastructure is either unused or under-utilized at any given time. These companies could make money by exposing their IT infrastructure via Web Services. Their (Web Services) API could then be used by other smaller players that service other companies.
In other words, just like Amazon.com and Google.com, the outsources could create a Web platform upon which others could build their own businesses. What these businesses could be is something that we will have to wait and find out. [They could be offering the same outsourcing services for the lower end of the market where the larger outsources' cost structure precludes their presence.]
But there is no reason why a company such as IBM could not re-package itself as a Web Platform and make money through its Web Services.
Some of this infrastructure is either unused or under-utilized at any given time. These companies could make money by exposing their IT infrastructure via Web Services. Their (Web Services) API could then be used by other smaller players that service other companies.
In other words, just like Amazon.com and Google.com, the outsources could create a Web platform upon which others could build their own businesses. What these businesses could be is something that we will have to wait and find out. [They could be offering the same outsourcing services for the lower end of the market where the larger outsources' cost structure precludes their presence.]
But there is no reason why a company such as IBM could not re-package itself as a Web Platform and make money through its Web Services.
Concurrency Pitfalls
- Avoid calling library functions. A library function is the classic case of "someone else's code." Unless the library function is documented to not take any locks, deadlock problems can arise.
- Avoid calling plug-ins. Clearly, a plug-in is "someone else's code."
- Avoid calling other callbacks, function pointers, functors, delegates, and so on. C function pointers, C++ functors, C# delegates, and the like can also fairly obviously lead to "someone else's code." Sometimes, you know that a function pointer, functor, or delegate will always lead to your own code, and calling it is safe; but if you don't know that for certain, avoid calling it from inside a critical section.
- Avoid calling virtual methods. This may be less obvious and quite surprising, even Draconian; after all, virtual functions are common and pervasive. But every virtual function is an extensibility point that can lead to executing code that doesn't even exist today. Unless you control all derived classes (for example, the virtual function is internal to your module and not exposed for others to derive from), or you can somehow enforce or document that overrides must not take locks, deadlock problems can arise if it is called while inside a critical section.
- Avoid calling generic methods, or methods on a generic type. When writing a C++ template or a Java or .NET generic, we have yet another way to parameterize and intentionally let "someone else's code" be plugged into our own. Remember that anything you call on a generic type or method can be provided by anyone, and unless you know and control all the types with which your template or generic can be instantiated, avoid calling something generic from within a critical section.
Enhancing Textual Information Retrieval
Information Retrieval (IR) is the science of searching for information in documents, searching for documents themselves, searching for metadata which describe documents, or searching within databases, whether relational stand-alone databases or hyper-textually-networked databases such as the World Wide Web.
IR encompasses the subject matter areas of data retrieval, document retrieval, information retrieval, and text retrieval, and each of these has its own bodies of literature, theory, praxis and technologies.
Automated IR systems are used to reduce information overload. Many universities and public libraries use IR systems to provide access to books, journals, and other documents. Web search engines such as Google, Yahoo search or Live Search (formerly MSN Search) are the most visible IR applications. Others are MS SQL English Query and such like systems that enable the construction of queries that use English.
A typical IR application, such as Yahoo or Google searches, could be initiated by a user that supplies a search string of text to the IR system. The IR system will then try to match the words in this textual search string, using computational linguistics techniques, against documents, or their metadata which describe those documents, or within any available databases. Once the system has executed its search algorithms against the target data sources (documents, metadata about those documents, data-stores, etc.) it will supply the user with a set of matches against the supplied search string.
It is often the case that the user is interested in retrieving information that are “about” his search string. On the other hand, at times the user could also be interested in finding information that is relevant to the search string but not necessarily about it. However, the “about-ness” search and the “relevance” search are 2 operations that do not commute (in the sense of Abstract Algebra). That is, the results of an “about” search followed by a “relevance” search are not, in general, the same as those of a “relevance” search followed by an “about” search. Often times, the users try to circumvent this by changing the order of the words in their search sentence, by using synonyms, and so on.
We can circumvent this limitation by the following 2 observations:
1. That the textual searches return different results based on the order of the words in the search string; .e.g. a Google search for “Black is lake” will return different (but over-lapping) results if one permutes the orders of the words. Thus one gets more results if one permutes the order of words in the search string and collects all the returned “hits”.
2. The second observation is that while word permutations in an English sentence (for example) could lead to meaningless sentences, the same loss of meaning does not obtain in all languages. In fact, in languages such as Russian, or Sanskrit, or Ancient Greek the sentence retains some meaning after permuting the word order. The reason is that these languages have preserved the Indo-European synthetic-inflexional structure. Inflection or inflexion is the modification or marking of a word to reflect grammatical (that is, relational) information, such as gender, tense, number or person.
In this approach, the search text string (say in English) is first translated automatically into one of the 3 languages mentioned above; i.e. Russian, Sanskrit, or Ancient Greek. An embodiment of this will be to use an existing Commercial-Off-The-Shelf translation (COTS) software that can translate automatically from and to the Russian language (an example is the package used in (www.babalefish.com) . Next, the translated Russian text will undergo automatic permutations of its word order. The system will maintain a list of each permuted search string of Russian text. The system will then translate back into English (from Russian in this embodiment) the permuted search strings. The resulting list of permuted English search strings is then supplied to the IR system. The results of all these permuted searches are then collected and collated and delivered to the user.
This approach may be immediately generalized to any language which is a member of the Indo-European family of languages; that is, almost all of the modern languages of North and South America, Europe, The Middle East, and South Asia.
IR encompasses the subject matter areas of data retrieval, document retrieval, information retrieval, and text retrieval, and each of these has its own bodies of literature, theory, praxis and technologies.
Automated IR systems are used to reduce information overload. Many universities and public libraries use IR systems to provide access to books, journals, and other documents. Web search engines such as Google, Yahoo search or Live Search (formerly MSN Search) are the most visible IR applications. Others are MS SQL English Query and such like systems that enable the construction of queries that use English.
A typical IR application, such as Yahoo or Google searches, could be initiated by a user that supplies a search string of text to the IR system. The IR system will then try to match the words in this textual search string, using computational linguistics techniques, against documents, or their metadata which describe those documents, or within any available databases. Once the system has executed its search algorithms against the target data sources (documents, metadata about those documents, data-stores, etc.) it will supply the user with a set of matches against the supplied search string.
It is often the case that the user is interested in retrieving information that are “about” his search string. On the other hand, at times the user could also be interested in finding information that is relevant to the search string but not necessarily about it. However, the “about-ness” search and the “relevance” search are 2 operations that do not commute (in the sense of Abstract Algebra). That is, the results of an “about” search followed by a “relevance” search are not, in general, the same as those of a “relevance” search followed by an “about” search. Often times, the users try to circumvent this by changing the order of the words in their search sentence, by using synonyms, and so on.
We can circumvent this limitation by the following 2 observations:
1. That the textual searches return different results based on the order of the words in the search string; .e.g. a Google search for “Black is lake” will return different (but over-lapping) results if one permutes the orders of the words. Thus one gets more results if one permutes the order of words in the search string and collects all the returned “hits”.
2. The second observation is that while word permutations in an English sentence (for example) could lead to meaningless sentences, the same loss of meaning does not obtain in all languages. In fact, in languages such as Russian, or Sanskrit, or Ancient Greek the sentence retains some meaning after permuting the word order. The reason is that these languages have preserved the Indo-European synthetic-inflexional structure. Inflection or inflexion is the modification or marking of a word to reflect grammatical (that is, relational) information, such as gender, tense, number or person.
In this approach, the search text string (say in English) is first translated automatically into one of the 3 languages mentioned above; i.e. Russian, Sanskrit, or Ancient Greek. An embodiment of this will be to use an existing Commercial-Off-The-Shelf translation (COTS) software that can translate automatically from and to the Russian language (an example is the package used in (www.babalefish.com) . Next, the translated Russian text will undergo automatic permutations of its word order. The system will maintain a list of each permuted search string of Russian text. The system will then translate back into English (from Russian in this embodiment) the permuted search strings. The resulting list of permuted English search strings is then supplied to the IR system. The results of all these permuted searches are then collected and collated and delivered to the user.
This approach may be immediately generalized to any language which is a member of the Indo-European family of languages; that is, almost all of the modern languages of North and South America, Europe, The Middle East, and South Asia.
Friday, November 9, 2007
Carbon Foot-print
You can calculate your carbon foot-print @ http://www.carbonfootprint.com/.
The company also provides commercial services for businesses.
The company also provides commercial services for businesses.
Wednesday, November 7, 2007
7 Tools for .NET
NDoc 2.0 (Free)
Using NDoc 2.0 you can easily generate a compiled help file or a Web site that documents your assemblies and presents them MSDN-style. The tool provides many features to customize the output and create professional-looking documentation. You can configure the headers, footer, title, copyright information and more with NDoc 2.0.
ReSharper 3.0 ($199)
This tool provides more than a dozen productivity enhancers in one utility. As its name suggests, the product began as a refactoring tool for C#. The latest version now supports VB.NET as well, and includes a total of 27 refactoring features for C# and 17 for VB.NET.
It has a feature called Stack Trace Explorer with which you can paste your stack trace into the explorer and the tool will render each stack frame as a link that you can click to jump straight to the corresponding code. The Find Usages feature is several times faster than Visual Studio's Find All References.
ReSharper also supports quick fixes. Suppose your assignment statement is missing a cast. ReSharper will offer to fix the code -- either by inserting the cast or changing the variable type. You can use ReSharper to format code according to your style guidelines with a single keystroke. ReSharper provides more intelligent IntelliSense than Visual Studio by filtering the selections based on the expected type. These features are just the tip of the iceberg -- if you'd like to increase your coding productivity, you need to try this tool.
Pattern Expert ($79)
This utility has a single purpose: to teach programmers how to implement design patterns in code. While this may classify more as a tutorial aid than a development tool, the fundamentals that it aims to teach ultimately motivate developers to write significantly better code, making it arguably the best kind of development tool available.
Reflector (Free)
Reflector lets you examine the contents of any .NET assembly. With it, you can browse an assembly's types, meta data, Intermediate Language (IL) instructions, resources and XML documentation. This makes it an excellent tool for studying the inner workings of any compiled assembly, including third-party tools and the .NET base class libraries themselves. These capabilities can help when troubleshooting problems that may lie in components for which no source code is available. Reflector works with all versions of the .NET Framework, and the latest version of this tool supports LINQ query expressions and other concepts introduced in .NET 3.5.
Outlook Redemption ($199)
This library offers the best object model yet for programmatically accessing mailboxes and their folders, mail items, appointments, etc. Perhaps the most compelling feature is its ability to bypass the warning dialogs raised by the Security Patch introduced since Office 2000 SP2. This makes it ideal for unattended service applications that need to talk to an Exchange store. Plus, Internet headers and extended properties (many of which are difficult or impossible to extract using MAPI or CDO) are all fully exposed and available through the object model.
RegexDesigner.NET (Free)
The best feature is the code generator, which will automatically create source code (in either VB.NET or C#) that implements your regular expression for either match, replace or split operations. The generated code for match operations includes check for match, get match, get matches, numbered groups and named groups. You can also compile your regular expressions directly to an assembly, which you can then reference from your application or reuse across multiple applications.
Fiddler (Free)
Fiddler is a debugging proxy, meaning that it situates itself in the pipeline so that all network traffic flows through it. It can therefore easily intercept and expose all HTTP requests and responses between client and server. The tool earns its name by giving you the ability to "fiddle" with the network conversation that's taking place. For example, you can build a handcrafted request, or drag/drop a previous session request, and then optionally tweak values before executing it. There's also support for breakpoints and single-step debugging, which pause and resume HTTP traffic while permitting edits during execution.
Developing for Facebook
Facebook was built with open source software and the Facebook Platform is designed in the same spirit.
The platform has a proprietary API, a Facebook Query Language (FQL) and a Facebook Markup Language. There’s no charge to use the tools.
The API uses a representational state transfer (REST)-based Web services interface; i.e. Facebook method calls are made over the Internet by sending HTTP GET or POST requests to REST server. With the API, one can add social context to an application by utilizing profile, friend, photo and event data. There are also plans to introduce support for simple object access protocol (SOAP) Web services in the future.
FQL employs a SQL-style interface for tapping Facebook data.; enabling developers to obtain more granular information about a set of items, instead of all of it, and gaining the ability to issue more detailed requests, which can result in fewer queries overall.
Facebook Markup Language is a subset of HTML that developers use to integrate their applications into the overall user experience, tapping points such as newsfeeds, profiles and other areas.
The company has set up a support and resource site at http://developers.facebook.com. It contains a wiki, documentation and official client libraries for PHP and Java. There are also a number of “unofficial” libraries on the site, including ones for ActionScript, .NET, Python, Ruby, Perl, ColdFusion, VB.NET and others.
There are millions of users at Facebook; surely there must be a way of making money here.
The platform has a proprietary API, a Facebook Query Language (FQL) and a Facebook Markup Language. There’s no charge to use the tools.
The API uses a representational state transfer (REST)-based Web services interface; i.e. Facebook method calls are made over the Internet by sending HTTP GET or POST requests to REST server. With the API, one can add social context to an application by utilizing profile, friend, photo and event data. There are also plans to introduce support for simple object access protocol (SOAP) Web services in the future.
FQL employs a SQL-style interface for tapping Facebook data.; enabling developers to obtain more granular information about a set of items, instead of all of it, and gaining the ability to issue more detailed requests, which can result in fewer queries overall.
Facebook Markup Language is a subset of HTML that developers use to integrate their applications into the overall user experience, tapping points such as newsfeeds, profiles and other areas.
The company has set up a support and resource site at http://developers.facebook.com. It contains a wiki, documentation and official client libraries for PHP and Java. There are also a number of “unofficial” libraries on the site, including ones for ActionScript, .NET, Python, Ruby, Perl, ColdFusion, VB.NET and others.
There are millions of users at Facebook; surely there must be a way of making money here.
Tuesday, November 6, 2007
Eric Verhulst on Parallel Programming
I enjoyed reading and learnt from Eric Verhulst's letter in the November 2007 issue of the Embedded System Design magazine.
There he writes:
"There will never be a real 'parallel' language. Actually what people mean is a compiler that turns a sequential program into a parallel one. In the best case, we will have something like the parallizing FORTRAN compilers. These compilers look for the loops and the split them over multiple processors. The issue is that one can never extract more parallelism than was originally put in the program. For a lot of scientific programs or even for some graphic applications there is some potential, but for most applications the potential is very limited. Even then, a lot more parallelism could be found if a parallzing compiler was not an exercise in reverse engineering. The original problems often have a lot of real-world parallelism. e.g. fluid dynamics code starts from a model where millions of small "voxels" and their interactions are integrated to obtain a sequential code. in the process, the "parallel" information gets lost."
And in the latest issue of SD Times Andrew Binstock has observed that:
"The vendors most concerned with promoting threading on the desktop—that is, chip vendors whose future is closely tied to capitalizing on multiple cores—are slowly but inevitably coming to a disappointing conclusion: Desktop developers are not interested in threading. And no amount of flag waving or promotion is going to persuade them to adopt threads."
I wonder then if it is worth learning Haskell, F#, or Erlang?
There he writes:
"There will never be a real 'parallel' language. Actually what people mean is a compiler that turns a sequential program into a parallel one. In the best case, we will have something like the parallizing FORTRAN compilers. These compilers look for the loops and the split them over multiple processors. The issue is that one can never extract more parallelism than was originally put in the program. For a lot of scientific programs or even for some graphic applications there is some potential, but for most applications the potential is very limited. Even then, a lot more parallelism could be found if a parallzing compiler was not an exercise in reverse engineering. The original problems often have a lot of real-world parallelism. e.g. fluid dynamics code starts from a model where millions of small "voxels" and their interactions are integrated to obtain a sequential code. in the process, the "parallel" information gets lost."
And in the latest issue of SD Times Andrew Binstock has observed that:
"The vendors most concerned with promoting threading on the desktop—that is, chip vendors whose future is closely tied to capitalizing on multiple cores—are slowly but inevitably coming to a disappointing conclusion: Desktop developers are not interested in threading. And no amount of flag waving or promotion is going to persuade them to adopt threads."
I wonder then if it is worth learning Haskell, F#, or Erlang?
Subscribe to:
Posts (Atom)
About Me
- Babak Makkinejad
- I had been a senior software developer working for HP and GM. I am interested in intelligent and scientific computing. I am passionate about computers as enablers for human imagination. The contents of this site are not in any way, shape, or form endorsed, approved, or otherwise authorized by HP, its subsidiaries, or its officers and shareholders.