Topology of a Data Product Team by Eric Broda via @TDataScience

The success of your data mesh journey will be dictated by your data product team structure rather than your technology choices. Here is what you need to know to setup your data product teams.

This is a great article that is well worth thinking about.

SLIDESHOW: 7 top challenges to working with data by David Weldon via @infomgmt

Data pros are dealing with a skyrocketing amount of data, created and gathered by ever-more devices. Here are the top challenges this is creating, according to a new study by Nexla.

From my own perspective these are a good list of pain points to the use of data. I would add to this  list:

1.. Data Sources - do you know the best place to get your data from - there could be better alternatives do get the data from.

2. System of Record - related to 1. make sure you understand where your data really comes from and if the data is clean and pure of has been altered in some way.

3. Change control - I've been using a systems data to feed in some of the data I was using, but they have missed it in their change control and I've suddenly had different or no data arrive.

4.  Data Management - are fields with the same name really the same?

Organizations gaining new benefits by automating data engineering by Jelani Harper via @infomgmt

A number of advancements have now decreased data preparation time while increasing the time available for exploration and applications.

I think there are several possible tools that enable users to do their own querying and this article talks about the one that the author is most familiar with.  Some organisations use Tableau, Pentaho or Power BI. I'm sure there are others I have not listed.

In the rush to big data, we forgot about search by @acoliver via @infoworld

In the cloud era, we need to look at search to be the glue that lets us find the data and analyse it together, no matter where it lives.

This is an important area that we all miss and it needs to be given a better focus as I'm sure we lose part of the benefit of putting the data or a system in the cloud because of the inadequate searching. If you think back to relational database design and then you think about searches then sometimes you even add indexes especially for common searches as part of tuning so we already know it needs attention.

The 3 most important data metrics for retaining customers by Matthew Tharp via @infomgmt

In an era where every customer is valuable, it’s important to use information to more deeply understand the most profitable and high-growth segments and tailor the business to them.

Some great suggestions on metrics as keeping customers is definitely cheaper than getting new ones.

The battle over bank customer data may finally be over by Penny Crosman via @infomgmt

Guidance from the CFPB and reduced friction between banks, fintechs and data aggregators are easing bank-fintech partnerships at Wells Fargo, Capital One and others.

I can see that this has a lot of benefits, but I can also see that there are a lot of potential drawbacks as it could become a security risk. I'm also interested in how it verifies that you are the right person.

SLIDESHOW: 14 top platforms for data integration by David Weldon via @infomgmt

Informatica, Talend and Oracle are among the leaders in this space, according to Gartner’s Magic Quadrant.

Interesting list and a few surprises for me too.

5 Data Management Mistakes to Avoid during Data Integration Projects by @mairabay via @hlsdk

In this article Canada based Maira Bay de Souza of Product Data Lake Technologies shares her view on data integration and the mistakes to avoid doing that.

I agree with her observations but feel that there needs to be more focus on the data that this article describes (although that could be because I spent so many years doing the detailed design for data integrations and loads on a data warehouse).

My thoughts are:

You need detailed documentation of the data at source, target and any processing in between. That documentation should cover formats, values, lookups, defaults, translations, timezones, currencies, master data location/values and anything else you can find.

You need to think about if you need to handle Slowly Changing Dimensions at all stages of the integration as they could impact your interface (I don't think they are something that only affects a Data Warehouse)

Data Exchange, Analytics Remain Out of Reach for Many Providers via ‎@HITAnalytics

Providers are still running into data exchange and interoperability roadblocks due to EHR shortcomings, leaving them unprepared to tackle value-based care.

Better design is needed to ensure that these kinds of issues don't happen in the future.

WEBINAR: Advanced analytics in the era of big data - 15 December 2016

Complimentary Web Seminar
December 15, 2016
2 PM ET/11 AM PT
Hosted by Information Management

Today’s advanced analytic environments are putting greater pressure on decision support infrastructures, creating a mandate for a better, more agile foundation to support them. If you are cracking under the pressure of delivering BI and analytics in a timely way, register for this webinar to hear experts share tips on delivering value to the business faster.

Learn about cutting-edge, data integration and database technologies that simplify the preparation and engineering of data – automating the means by which it is integrated, transformed, and managed – along with the process of manipulating and analyzing data at massive scale.

Topics to be covered include:

• Technologies that enable the rapid and agile integration and processing of data
• How to simplify and accelerate the efforts of data scientists and business analysts
• The role of big data in advanced analytics

Featured Presenters:

Moderator: Eric Kavanagh Information Management	Speaker: Donald Farmer TreeHive Strategy	Speaker: Shawn Rogers Statistica	Speaker: Imad Birouty Teradata	Speaker: Michael Whitehead WhereScape

Sponsor Content From:

Register here

To Achieve Advanced Analytics, Start with Big Data Integration by John Thielens via @infomgmt

Big data requires new forms of processing and thus, innovative technology to support and create enhanced decision-making and greater insights. It’s no easy task given the scale at which we’re doing business today.

I know myself from my experience organisations have many disparate systems with the same data in fields with different names, different formats, etc. Integrating all this data is an art which needs careful consideration. In a perfect world everything would be designed to be the same in all the various data source but the world is not perfect.  In order to get insights and value out of all your data you need to integrate it first.

Operational data governance: Who owns data quality problems? by David Loshin via @SASsoftware

Data integration teams often find themselves in the middle of discussions where the quality of their data outputs are called into question. Without proper governance procedures in place, though, it's hard to address these accusations in a reasonable way. Here's why.

I completely agree with the second point he makes at the end of the article - far too many times I've worked on a data warehouse project where that team is concerned about the data quality far more than the source system team - to my mind that is wrong.

SLIDESHOW: The 14 Top Data Integration Companies via @infomgmt

Gartner Group has just released its “2016 Gartner Magic Quadrant for Data Integration Tools.” Here’s a look at the top 14 companies, and what each has to offer.

I particularly like Informatica and Oracle as I have more experience of them.

WEBINAR: Turn Big Data into Smart, Trusted Assets - 3 May 2016

Turn Big Data into Smart, Trusted Assets

DATE: Wednesday, May 3, 2016

TIME: 1:00 PM ET

With exploding data volumes, leveraging the distributed computing power of Hadoop is more necessary than ever. Learn how Pentaho Data Integration can quickly and effectively take full advantage of the architecture to deliver powerful data integration and business analytics, all while using Melissa Data’s Data Quality tools to deliver clean, verified and trusted assets and analytics. 

• Learn how to make quick and easy integration and analysis of large data sets, leveraging Pentaho Data Integration
• Leverage the power of Hadoop with your data quality initiatives
• Learn how to transform your data into clean, reliable assets

FEATURED SPEAKERS:

Robert Smith, Enterprise Sales Engineer, Pentaho

Patrick Bayne, Data Quality Analyst and Software Engineer, Melissa Data

Register here

Growing Complexity of Data Integration, Governance Could Harm Companies via @infomgmt

Growing Complexity of Data Integration, Governance Could Harm Companies by Bob Violino via +Information Management  - The increasing complexity of the enterprise resource planning (ERP) application portfolio is driving the need for a “defined postmodern” application integration strategy, according to Gartner Inc.

I think that hybrid environments increase the need for good data integration and govenance - if we don't do that we will lose control which will make the data and any results pretty worthless.

Five Patterns of Big Data Integration via @LakshmiLJ

Five Patterns of Big Data Integration via @LakshmiLJ - in this article by Lakshmi Randall on +Datafloq she discusses that as our reliance on Hadoop and Spark for data management, processing and analytics grows, data integration strategies should evolve to exploit big data platforms in support of digital business, Internet of Things (IoT) and analytics use cases.

I can imagine many are stuck thinking of the "old" was of integrating data and many processes and standards have not bee updated nor adapted for this new world of data integration.

Top Three Best Practices for Data Integration Deployment via @BigData_Review

According to the IDC, data stored in enterprise applications is expected to grow by 50 percent each year to 40 zettabytes by 2020. Here are three best practices for Data Integration from Solutions Review.

A good reminder of what should be considered when thinking about your own data.

Why Business Leaders are Clueless about Data Integration via @nyike

Great blog from Isaac Sacolick.  Data Integration is key to get right and often ignored. I have personally spent many years designing interfaces to load and subsequently integrate data.  Without fail, all were reliant on no changes being made that you didn't know about or could be part of the testing and implementation of.  Frequently that was not the case and was therefore a point of failure.

Data Integration Talent: It Makes or Breaks Big Data & Cloud

This  article on +Matrix IBS discusses the importance of Data Integration.

As someone who specialises in integrating data I can definitely agree with that.

Data integration essential for maximising business value and performance with Big Data

This article by +Attunity, Inc. discusses the fact that one of the key problems with all the data that organisations now have is that they need to be integrated in order to provide some value.

I would so recommend looking at this page on the +Talend  website as their open source data integration software can be used with Hadoop.

This Whitepaper from +TDWI  looks at it from the Agile Information and Integration Governance (IIG) angle.

What all these links tell us is that data needs to be correctly integrated to provide value and that you will be more successful the greater your level of IIG maturity.