On-demand analysis is increasingly needed as we strive to get a competitive edge over rivals or adhere to auditing or regulations.
In this TDWI White Paper Cirro describe scenarios for providing the ability to do on-demand analysis and the potential solutions and costs involved.
Cirro's Data Hub enables you to write a single query that then splits into queries per data source and then knit them back together. A bit like in Teradata when a query is shared across all amps and then brought together by the controlling amp at the end.
Many people confuse on-demand analysis and real-time data in their mind.
Real-time data can be very expensive to provide for analysis and thought should be taken into if it is necessary. This TDWI Article David Stodder discusses when it is the right time for using data real-time. I think his point 3 is very important (Don't assume that application business rules can handle real time).
Saturday, 31 May 2014
Friday, 30 May 2014
T-SQL- how to find who owns a temp table
In this article on the SQL Performance website Aaron Bertrand explains for several versions of SQL SERVER how to work out ownership of a temp table.
Thursday, 29 May 2014
T-SQL - user defined functions - 10 questions you were too shy to ask
In this post on Simple Talk written by Robert Sheldon he goes through some simple solutions to those questions (with code examples) on user defined functions.
I particularly like that a non-deterministic function can be used in a function which I had never realised but makes sense when you think about it.
I particularly like that a non-deterministic function can be used in a function which I had never realised but makes sense when you think about it.
Wednesday, 28 May 2014
Informatica,Virtual Data Machines and Vibe
In this Information Management blog entry by Bruce Guptill he talks about Informatica and their view of MDM.
In this IM blog it mentions Vibe from Informatica. This is what they describe as a Virtual Data Machine (VDM) where the instruction and specification (business logic) are stored separately from where it will be executed. This will enable definitions to be standardised and implemented on any machine or platform. Their standard MDM is described as Multidomain MDM so I guess Vibe is just an extension of that idea.
In this IM blog it mentions Vibe from Informatica. This is what they describe as a Virtual Data Machine (VDM) where the instruction and specification (business logic) are stored separately from where it will be executed. This will enable definitions to be standardised and implemented on any machine or platform. Their standard MDM is described as Multidomain MDM so I guess Vibe is just an extension of that idea.
Data protection for big data
In these two white papers they look at data protection from two different angles. But the overall consensus is that is must be done, should be done properly and is a business necessity with all the data protection legislation worldwide. Big data should not make any difference to the necessity to do something about data protection, just that the method might be slightly different due to the size of that data.
In this TDWI white paper by IBM on data protection for big data they advocate putting data protection in place from the start which is sensible considering the cost of doing anything later is always higher. Then in this Information Management article by Maria Aspan she discusses why banks still struggle with Big Data. I have to agree that I suspect it is partially privacy concerns but also a large part of disparate systems as banks merged in the past.
In this TDWI white paper by IBM on data protection for big data they advocate putting data protection in place from the start which is sensible considering the cost of doing anything later is always higher. Then in this Information Management article by Maria Aspan she discusses why banks still struggle with Big Data. I have to agree that I suspect it is partially privacy concerns but also a large part of disparate systems as banks merged in the past.
Tuesday, 27 May 2014
7 deadly sins of database design
In this white paper in Information Management sponsored by Embarcadero it goes through what they consider to be the 7 deadly sins of database design.
Whilst I do roughly agree with them on their 7 I would either add or update the list:
5. Data quality can be implemented using alternatives to a foreign key or check constraint on the database. If you are following an object oriented approach to the data you can create a common method that ensures that quality that can be enforced without adding database objects that coulld slow down any update/insert into that table on the database.
8. Changes in company documentation or modelling standards over time often result in mismatched levels or standards for each artefact. It would be great if there were time allowed to update older documentation as standards change. If that was impractical then any project plan touching those items should include time to update artefacts to the new standard.
Whilst I do roughly agree with them on their 7 I would either add or update the list:
5. Data quality can be implemented using alternatives to a foreign key or check constraint on the database. If you are following an object oriented approach to the data you can create a common method that ensures that quality that can be enforced without adding database objects that coulld slow down any update/insert into that table on the database.
8. Changes in company documentation or modelling standards over time often result in mismatched levels or standards for each artefact. It would be great if there were time allowed to update older documentation as standards change. If that was impractical then any project plan touching those items should include time to update artefacts to the new standard.
Monday, 26 May 2014
Spurious Correlations website
The Spurious Correlations website by Tyler Vigen is a great site for showing how you must think carefully before you link data together and show a correlation as you can to that for completely unrelated data.
The one on the front of his website at the time I wrote this entry shows US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation. Of course it doesn't and shouldn't if you think about it, but they look like they do.
The one on the front of his website at the time I wrote this entry shows US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation. Of course it doesn't and shouldn't if you think about it, but they look like they do.
Subscribe to:
Posts (Atom)