https://www.datalumen.eu/wp-content/uploads/2018/10/accounting-analytics-balance-209224O.jpg17072560Dimitri Maesfranckxhttps://www.datalumen.eu/wp-content/uploads/2019/06/datalumen_color_V1.1-340-156B.pngDimitri Maesfranckx2022-04-08 14:54:282022-05-25 16:11:58CALCULATING DATA GOVERNANCE ROI
Data Virtualization is definitely on the rise. At its Data and Analytics Summit in London, Gartner was projecting accelerated data virtualization adoption for both first-time and expanded deployments. Besides market analysts, we also see high demand and can confirm this being one of the hottest data solutions. But what are the top uses cases for data virtualization?
Logical Data Warehouse Architecture (LDW)
Traditional data warehouses entered on repositories are no longer sufficient to support today’s complex data and analytics landscape. The logical data warehouse (LDW) combines the strengths of traditional repository warehouses with alternative data management and access strategies to improve your agility, accelerate innovation, and respond more efficiently to changing business requirements.
Challenges include:
A data services approach that separates data access from processing, processing from transformation, and transformation from delivery
Diverse analytic tools and users
Diverse data types and sources including traditional data repositories, distributed processing (big data), virtualized sources, and analytic sandboxes
Unified business ontologies that resolve diverse IT taxonomies via common semantics
Unified information governance including data quality, master data management, security, and more
Service level agreement (SLA) driven operationalization
Data Virtualization provides a virtualization-centric LDW architecture solution.
With Data Virtualization you can:
Access any source including traditional data repositories, distributed processing (big data), virtualized sources, and analytic sandboxes, both onpremises and in the cloud
Model and transform data services quickly, in conformance with semantic standards
Deliver data in support of a wide range of use cases via industry-standard APIs including ODBC, JDBC, SOAP, REST, and more
Share and reuse data services across many applications
Automatically allocate workloads to match SLA requirements
Align data access and use with enterprise security and governance requirements
With Data Virtualization you get:
One logical place to go for analytic datasets regardless of source or application
Better analysis from broader data access and more complex transformations
Faster analysis time-to-solution via agile data service development and reuse
Higher quality analysis via consistent, well-understood data
Higher SLAs via loose-coupling and optimization of access, processing and transformation
Flexibility to add or change data sources or application consumers as required
More complete and consistent enterprise data security and governance
Logical Data Lake (LDL)
The idea of having all of your data stored and available in a data lake sounds like a wonderful idea to everyone. Finally the promise of Big Data can become reality. But what is the reality behind implementing a data lake? Virtually all organizations already have numerous data repositories – data warehouses, operational data stores, data stored in files, etc. It is impossible to load all of this data into a data lake and give everyone access to it. Next to data volume, You should also think about other elements like different data formats, data quality and even data security.
All this complexity however doesn’t mean that you should forget about a data lake. Using Data Virtualization, it is possible to leave the data in its current environment and create a virtual or better a logical data lake.
Challenges include:
Move all the data into a single location
Move all the data in a timely manner to that single location
Deliver the data as integrated data
Deliver fresh data
Where to store the date and how limit the impact when this storage mechanism needs to be changed
Avoid and limit the impact of the ever changing technologies
Data Virtualization lets you:
Isolate your data consumers from the underlying data architecture
Be flexible in changing the underlying architecture without impacting your data consumers
Deliver fresh data without the need to move it upfront
With Data Virtualization you get:
Fresh data for your data consumers
Agile data delivery in the shape the data consumers like to have it
Deliver data independent whether it is located in traditional systems, multiple big data systems or a combination of all of these
Agility to change the data architecture without impacting the data consumers
Prototyping For Physical Integration
Physical integration is a proven approach to analytic data integration; However, long lead times associated with physical integration — on average 7+ weeks according to TDWI — can delay realizing business value. Further, physical integration requires significant data engineering efforts and a complex software development lifecycle.
Challenges include:
Requirements. Business requirements are not always clear at the start of a project and thus can be difficult for business users to clearly communicate.
Design. Identifying and associating new mappings, new ETLs, and schema changes is complex. Further, current data engineering staff may not understand older schemas and ETLs. This makes detailed technical specifications a key requirement.
Development. Schema changes and ETL builds are required prior to end user validation. Resultant rework cycles often delay solution delivery.
Deployment. Modifying existing warehouse / data mart schemas and ETLs can be difficult and/or risky.
Data Virtualization lets you:
Interactively refine requirements, and based on actual data, build virtual data services side-by-side with business users.
Quickly deploy agreed datasets into production to meet immediate business needs.
Invest additional engineering efforts on physical integration later, only if required.
If required, use mappings and destination schema within the proven dataset as a working prototype for physical integration ETLs and schema changes.
Once physical integration is tested, transparently migrate from virtual to physical without loss of service.
With Data Virtualization you get:
Faster time-to-solution than physical integration, and accelerated business benefits
Less effort spent on upfront requirements definition and technical specification
The right level of data engineering required to meet requirements, while avoiding unnecessary over-engineering
Less disruption of existing physical repositories, schemas, and ETLs
Data Access / Semantic Layer For Analytics
Vendor-specific analytic semantic layers provide specialized data access and semantic transformation capabilities that simplify your analytic application development.
However, these vendor-specific semantic layer solutions have limitations including:
Delayed support of new data sources and types
Inability to share analytic datasets with other vendor’s analytic tools
Federated query performance that is not well optimized
Limited range of transformation capabilities and tools
Data Virtualization provides a vendor-agnostic solution to data access / semantic layer for analytics challenges.
Challenges include:
Access any data source required
Model and transform analytic datasets quickly
Deliver analytic data to a wide range of analytics vendor tools via industrystandard APIs including ODBC, JDBC, SOAP, REST, and more
Share and reuse analytic datasets across multiple vendors’ tools
Automatically optimize queries
Conform analytic data access and delivery to enterprise security and governance requirements
With Data Virtualization you get:
One place to go for analytic datasets regardless of analytic tool vendor
Better analysis from broader data access and more complex transformations
Lower costs, with reuse of analytic datasets across diverse analytic tools and users
Faster query performance
Greater analytic data security and governance
Data Preperation
Self-service data preparation has proven to be a great way for business users to quickly transform raw data into more analytic friendly datasets. However, some agile data preparation needs require data engineering skill and higher-level integration capabilities.
Challenges include:
Support for increasingly diverse and distributed data sources and types
Limited range of transformation capabilities and tools
Constraints on securing, governing, sharing, reusing, and productionizing prepared datasets
Data Virtualization provides an agile data preparation solution for data engineers that complements business user data preparation tools.
Data Virtualization lets you:
Interactively refine requirements and prepare datasets with business users based on actual data
Prepare datasets that may require complex transformations or highperformance queries
Leverage existing datasets when preparing new datasets
Quickly deploy prepared datasets into production when appropriate
Align data preparation activities with enterprise security and governance requirements
With Data Virtualization you get:
Rapid, IT-grade datasets that meet analytic data needs
The right level of data engineering required to meet requirements, while avoiding unnecessary over-engineering
Less effort spent productionizing datasets
More complete and consistent data security and governance
Virtual Operational Data Stores (VODS)
Physical operational data stores (ODS) have proven a useful compromise that balances operational data access needs with operational system SLAs. However, replicating operational data in an ODS is not without it costs.
Challenges include:
Significant development investments for ODS set up, and for integration projects that move data to them.
Higher operating costs for managing the associated infrastructure.
Integration workloads on the operational system.
Often the operational source is not resource constrained, or operational queries may be light enough to not create significant workloads.
When operational data is in an ODS, it may still require further transformations to make it useful for diverse analysis needs.
Data Virtualization lets you:
Access any operational data or other sources as required
Model and transform operational datasets quickly
Deliver data to a wide range of operational applications via industry-standard APIs, protocols, and architectures including ODBC, JDBC, SOAP, REST, and more
Share and reuse analytic datasets across applications
Reduce the impact on operational sources via query optimization and intelligent caching
Conform operational data access and delivery to enterprise security and governance requirements
With Data Virtualization you get:
One virtual place to go for operational data
Better analysis from broader data access and more flexible transformations
Lower costs due to less replicated data maintained in physical ODSs
More than good enough query performance without impacting operational system SLAs
Data Hub Enablement
The data hub is logical architecture that enables data sharing by connecting producers of data (applications, processes, and teams) with consumers of data (other applications, processes, and teams). Master data hubs, logical data warehouses, customer data hubs, reference data stores, and more are examples of different kinds of data hubs. Data hub domains might be geographically focused, business process-focused, or application-focused.
Challenges include:
The data hub provision data to and receive data from analytic and operational applications
Hub data is governed and secure
Data flows into and out of the hub are visible
Data Virtualization data hub solution delivers these requirements.
Data Virtualization lets you:
Introspect sources and identify potential data hub entities and relationships
Access any data hub data source
Model and transform data hub datasets
Deliver data hub datasets to diverse analytic and operational applications via industry-standard APIs, protocols, and architectures including ODBC, JDBC, SOAP, REST, and more
Share and reuse data hub datasets across multiple applications
Conform data hub access and delivery to enterprise security and governance requirements
With Data Virtualization you get:
A complete solution for data hub implementations
Better analysis and business processes via consistent use of data hub datasets
Higher analytic and operational application quality via consistent use of data hub datasets
Greater agility when adding or changing data hub datasets
Complete visibility into data hub data flows
End-to-end data hub security and governance
Registry-Style Master Data Management (MDM)
Master Data Management (MDM)is an essential capability. Analyst firms such as Gartner have identified four MDM implementation styles (consolidation, registry, centralized, and coexistence) that you can deploy independently or combine to help enable successful MDM efforts.
Challenges include:
Access to master and reference data from diverse sources
A cross-reference table (index) that reconciles and links related master data entities and identifiers by source
Data services that expose the cross-reference table to analytic and operational applications that require master data from one or more sources
Data federation that leverages the cross-reference table when querying detailed data associated with master entities
Data Virtualization is a poven technology for registry-style MDM solutions.
Data Virtualization lets you:
Introspect sources and identify potential master data entities and relationships
Build a physical master data registry that relates and links master data across sources
Cache registry copies adjacent to MDM user applications to accelerate frequent MDM queries
Combine master, detail, and non-master data to provide more complete 360-degree views of key entities
With Data Virtualization you get:
A complete solution for registry-style MDM implementations
Better analysis via more complete views of master data entities across sources
Higher analytic and data quality via consistent use of master and reference data
Faster query performance and less disruption to master data sources
Greater agility when adding or changing master and reference data sources
Legacy System Migration
New technology provides more advanced capabilities and lower cost infrastructure. You want to take advantage. However, migrating legacy data repositories to new ones or legacy applications to new applications technology is not easy.
Challenges include:
Business continuity requires non-stop operations before, during, and after the migration.
Applications and data repositories are often tightly coupled making them difficult to change.
Big bang cutovers are problematic due to so many moving parts.
Too often, testing and tuning only happen after the fact.
Data Virtualization provides a flexible solution for legacy system migration challenges.
Data Virtualization lets you:
Create a loosely coupled, middle-tier of data services that mirror as-is data access, transformation, and delivery functionality
Test and tune these data services on the sidelines without impacting current operations
Modify the as-is data services to now support the future-state application or repository, then retest and retune
Migrate the legacy application or repository
Implement future-state data services to consume or deliver data to and from the new application or repository
With Data Virtualization you get:
To take advantage of new technology opportunities that can improve your business and cut your costs
Loose coupling you need to divide complex migration projects into more manageable phases
Less risk by avoiding big bang migrations
Reusable data services that are easy to modify and extend for additional applications and users
Application Data Access
Your applications run on data. However, application data access can be difficult.
Challenges include:
The need to understand and access increasingly diverse and distributed data sources and types
Difficulty in sharing data assets with other applications
Federated query performance that may require optimization
Complex transformations that may require specialized tools and techniques
Complex data and application security requirements that need to be enforced
Data Virtualization provides a powerful solution to these application data access challenges.
Data Virtualization lets you:
Access any data source required
Model and transform application datasets quickly
Deliver data to a wide range of applications development tools via industry standard APIs, protocols, and architectures including ODBC, JDBC, SOAP, REST, and more
Share and reuse application datasets across multiple analytic and operational applications
Automatically optimize queries
Conform data access and delivery to enterprise security and governance requirements
Cloud Data Sharing
With the rise of cloud-based applications and infrastructure, more data than ever resides outside your enterprise. As a result, your need to share data across your cloud and enterprise sources has grown significantly.
Challenges include:
The need to understand and access increasingly diverse cloud data sources and APIs
Diverse data consumers, each with their own data needs and application technologies
Complex transformations that may require specialized tools and techniques
Wide-area network (WAN) query performance that may require optimization
Complex cloud data security requirements that need to be enforced
Data Virtualization provides a powerful solution for these cloud data sharing challenges.
Data Virtualization lets you:
Access any cloud data source
Model and transform cloud datasets quickly
Deliver cloud data to a wide range of applications development tools via industry-standard APIs, protocols, and architectures including ODBC, JDBC, SOAP, REST, and more
Share and reuse cloud data across multiple applications
Automatically optimize queries and apply caching to mitigate WAN latency
Align data access and delivery to conform with enterprise and cloud data security and governance requirements
With Data Virtualization you get:
One place to go for cloud and enterprise data
Better applications from broader cloud data access and more complex transformations
Lower costs due to dataset reuse across diverse applications
Faster query performance
Greater cloud data security and governance
Interested in Data Virtualization?
Would you like to know how Datalumen can also help you understand how your organization can benefit from using Data Virtualization? Contact us and start our data conversation.
https://www.datalumen.eu/wp-content/uploads/2018/07/Datalumen-Data-Virtualization-Use-Cases.jpg17072560Dimitri Maesfranckxhttps://www.datalumen.eu/wp-content/uploads/2019/06/datalumen_color_V1.1-340-156B.pngDimitri Maesfranckx2022-01-17 21:53:282022-05-25 16:15:33DATA VIRTUALIZATION: TOP USE CASES THAT MAKE A DIFFERENCE
Last week TIBCO NOW 2020, a first-ever all-digital TIBCO NOW Connected Experience, was organized. A 3 day event stacked with the latest and the greatest about the TIBCO end-to-end offering. According to TIBCO close to 6000 attendees from over 80 countries and representing almost every industry tuned in. The main theme was TIBCO’s intention to propel Sustainable Innovation with technology frameworks, supported by proven solutions, best practices and techniques that support customers’ need for continual business reinvention.
Day 1-2-3
Day 1 featured two special guests, Jen Sey, SVP & CM for Levi Strauss, and Charlotte Yarkoni, CVP of Commerce & Ecosystems at Microsoft. Both executives spoke about how they use innovation and data to meet customers where their needs are.
Day 2 built upon the discussions of why sustainable business practices and data-centric tools are critical to very practical discussions about how organizations can build a foundation for sustainable innovation. ”Data is the common element,” explained Dan Streetman, TIBCO’s CEO. “How data connects systems, people, and devices. How we manage, unify, and bring order to that data and ultimately how we build on that data foundation to make faster, smarter decisions is how you build a framework for sustainable innovation.”
Day 3 was all about delivering on the promise of sustained innovation and was led by TIBCO’s Head of Customer Excellence, Jeff Hess. He explained that it all starts with putting your customers first. You need to constantly ask “How can we make life better for our customers?” It’s a powerful clarifier in a world of competing priorities. By asking this question every day it’s the only way to ensure you can deliver on the promise of sustained innovation.
What’s new in the TIBCO data management corner?
TIBCO is an end-to-end software vendor and TIBCO Now 2020 was used to showcase all of the newer existing but also upcoming technology. Given our partnership and our field of expertise as a data management & data governance solution provider, we will zoom into what’s new in the TIBCO Unify domain and the most interesting data management topics.
TIBCO Any Data Hub.
One of the bigger announcements was the launch of TIBCO Any Data Hub framework. This is basically a host of technologies and proven methodologies to help customers sustain innovation. Being hyperconnected, using artificial intelligence and data science and empowering everyone in an organization with data is how you can innovate. Supporting TIBCO’s “better together” strategy, the TIBCO Any Data Hub framework supports tighter integration across its portfolio, including integration with TIBCO Cloud Integration, TIBCO Cloud Mashery, TIBCO Cloud Metadata Management and TIBCO EBX. This enables customers to build out data-as-a-service architectures and an all-in-one approach for data definitions, one data catalog, one point of access, one security model and one governance system. However it’s an open framework so it can also quiet easily hook into your existing landscape and get setup with the data management components you already have in place.
TIBCO Data Virtualization.
A core component of TIBCO’s Any Data Hub is TIBCO’s market-leading data virtualization platform. TIBCO Data Virtualization latest 8.3 release supports now a significantly expanded set of data adapters. With TIBCO Data Virtualization, you can now connect to over 300+ data sources including support for streaming sources such as Apache Kafka, OSIsoft, and OPC Unified Architecture. In addition, it is now available on the Microsoft Azure Marketplace, along with the current AWS Marketplace offering and has a Dockerfile option. Besides info on the new 8.3 version we also received insight in the roadmap and what TIBCO intends to implement in the future TDV.
TIBCO EBX & Cloud Metadata. Next to that there was also the necessary focus on the key Master Data Management (MDM) component, TIBCO EBX (former Orchestra Networks MDM that got acquired), which still manages to dominate the MDM market. Linked to that and better integrated is the second generation of the recently introduced TIBCO Cloud Metadata and step by step starts to extend the platform capabilities in the meta data management domain.
Ready for your TIBCO NOW binge-watching?
TIBCO NOW 2020 has finished, but be aware that you can still watch everything on-demand through October 9 2020. TIBCO is leaving registration open after the event, so if you haven’t already, you have one last chance to register.
Sessions worthwhile checking out:
Empower Your Business with Any Data from Any Location at Any Speed. Watch ▶
Build a virtual data layer that integrates your enterprise data silos. Watch ▶
Govern All Your Master and Reference Data. Watch ▶
Discover, Catalog, and Govern All Your Enterprise Metadata. Watch ▶
OPG: Billion-dollar Nuclear Programs on Time and on Budget with MDM. Watch ▶
What’s new about TIBCO EBX & TIBCO Cloud Metadata. Watch ▶
Did you miss any of these sessions? Do you have a specific data question? Would you like to have a briefing or a demo for your team? Let us know and we are happy to setup a one-on-one.
https://www.datalumen.eu/wp-content/uploads/2020/10/Datalumen-TIBCONOW2020-scaled.jpg17082560Datalumenhttps://www.datalumen.eu/wp-content/uploads/2019/06/datalumen_color_V1.1-340-156B.pngDatalumen2020-10-02 16:43:512020-10-02 18:46:26TIBCO NOW 2020 – KEY DATA MANAGEMENT NEWS
Summer is here and the longer days it brings means more time available to spend with a ripping read. That’s how it ideally works at least. We selected 3 valuable books worth your extra time.
The Chief Data Officer’s Playbook
The issues and profession of the Chief Data Officer (CDO) are of significant interest and relevance to organisations and data professionals internationally. Written by two practicing CDOs, this new book offers a practical, direct and engaging discussion of the role, its place and importance within organisations. Chief Data Officer is a new and rapidly expanding role and many organisations are finding that it is an uncomfortable fit into the existing C-suite. Bringing together views, opinions and practitioners experience for the first time, The Chief Data Officer’s Playbook offers a compelling guide to anyone looking to understand the current (and possible future) CDO landscape.
Data Virtualization: Going Beyond Traditional Data Integration to Achieve Business Agility
Data Virtualization: Going Beyond Traditional Data Integration to Achieve Business Agility, the first book ever written on the topic of data virtualization, introduces the technology that enables data virtualization and presents ten real-world case studies that demonstrate the significant value and tangible business agility benefits that can be achieved through the implementation of data virtualization solutions. The book introduces the relationship between data virtualization and business agility but also gives you a more thorough exploration of data virtualization technology. Topics include what is data virtualization, why use it, how it works and how enterprises typically adopt it.
Simon Sinek started a movement to help people become more inspired at work, and in turn inspire their colleagues and customers. Since then, millions have been touched by the power of his ideas, including more than 28 million who’ve watched his TED Talk based on ‘Start With Why’ — the third most popular TED video of all time. Sinek starts with a fundamental question: Why are some people and organizations more innovative, more influential, and more profitable than others? Why do some command greater loyalty from customers and employees alike? Even among the successful, why are so few able to repeat their success over and over?
People like Martin Luther King, Steve Jobs, and the Wright Brothers had little in common, but they all started with Why. They realized that people won’t truly buy into a product, service, movement, or idea until they understand the Why behind it. ‘Start With Why’ shows that the leaders who’ve had the greatest influence in the world all think, act, and communicate the same way — and it’s the opposite of what everyone else does. Sinek calls this powerful idea The Golden Circle, and it provides a framework upon which organizations can be built, movements can be led, and people can be inspired. And it all starts with Why.
We’re giving away 50 copies of ‘Data Virtualization: Going Beyond Traditional Data Integration to Achieve Business Agility’. Want to win? Just complete the form and cross your fingers. Good luck!
Winners are picked randomly at the end of the giveaway. Our privacy policy is available here.
https://www.datalumen.eu/wp-content/uploads/2018/08/Datalumen-Summer-Reading-2018-ORIG-Covers.png11142560Datalumenhttps://www.datalumen.eu/wp-content/uploads/2019/06/datalumen_color_V1.1-340-156B.pngDatalumen2018-08-21 14:29:422020-09-22 07:52:32SUMMER READING TIP
According to this Forrester report, the enterprise data virtualization market has expanded along multiple dimensions – customer adoption, more industries, more use cases, new players, and acquisitions.
More customer adoption– Forrester states customer adoption of data virtualization has been gaining momentum. In 2017, Forrester profiled 2,106 global technology decision makers for its Data Global Business Technographics Data and Analytics Survey, and found that “…56% of global technology decision makers in our 2017 survey tell us they have already implemented, are implementing, or are expanding or upgrading their implementations of DV technology, up from 45% in 2016.”
More industries– Forrester states that in its early years, data virtualization was primarily used in financial services, telecom, and government sectors. In the last 5 years, however, Forrester has found significant adoption of DV in insurance, retail, healthcare, manufacturing, oil and gas, and eCommerce verticals as well.
More use cases– Further, Forrester found that among the customers who have been using data virtualization, the deployment has increased from single-use case, primarily customer analytics, to a broader enterprise-wide use involving multiple use cases such as internet of things, fraud detection, and integrated insights.
Acquisitions– In signs that the data virtualization market is maturing, TIBCO Software recently acquired Cisco Information Server, thus entering the data virtualization market.
We think all these data points are significant indicators that the data virtualization market is a healthy, growing market that is reaching maturity.
Data Virtualization Poised for Further Growth Pushed Forward by Leaders
Forrester expects the data virtualization market to grow further “because more enterprise architecture (EA) professionals see data virtualization as critical to their enterprise data strategy.” It says that these EA Pros are looking to support more complex data virtualization deployments. To satisfy such needs, the leaders featured in the report provide high-end scale, security, modeling, and broad use case support with their mature product offerings. “The leaders we identified offer large and complex deployments, and they support a broader set of use cases and more mature data management capabilities,” Forrester says. It is worth noting that four of the past five leaders retained their Leaders positions, while one vendor slipped into the Strong Performers.
Read the Complete Report
The Forrester Wave: Enterprise Data Virtualization, Q4 2017 is a must read for enterprise architecture (EA) professionals. According to Forrester, “Enterprise data virtualization has become critical to every organization in overcoming growing data challenges. These platforms deliver faster access to connected data and support self-service and agile data-access capabilities for EA pros to drive new business initiatives.”
Business intelligence & analytics today have dramatically shifted from the traditional IT-driven model to a modern self-service approach. This is due to a number of changes, including the fact that the balance of power has steadily shifted from IT to the business, and also the fact that the business community has new access to more innovative technologies that give them powerful analytical and visualization capabilities (e.g.Tableau, …). This increased use and capability has put the business in the driver seat of much front-end BI decision-making.
In order to help your business community continue to increase its self-service capabilities, there is one important, but often-overlooked item: Many implementations fail to realize their full potential because they fall into the trap of building out just the proverbial shop window, and forgetting the actual shop! It is just as important to add increased accessibility and flexibility to the underlying data-layer (and ease the access, discovery, andgovernance of your data), as it is to provide users the front-end thru powerful analytics and visualization capabilities.
With respect to self‐service analytics, four phases can be identified in the market. These also typically mirror how analytics are implemented in many of companies. The following diagram describes in four phases howdata virtualizationcan strengthen and enrich the self‐service data integration capabilities of tools for reporting and analytics:
THE NEED FOR DATA PREPARATION AND DATA VIRTUALIZATION
To support both IT-driven and Business-driven BI, two techniques are required: data preparation, and data virtualization. There are a many scenarios where you can use these techniques to strengthen and speedup the implementation of self‐service analytics:
Using data virtualization to operationalize user‐defined data sets
Using data virtualization as a data source for data preparation
Using data virtualization to make data sets developed with data preparation available for all users
To learn about how to succeed in your data journey, feel free tocontact us. More info about our full spectrum of data solutions is also available on theDatalumen website.
https://www.datalumen.eu/wp-content/uploads/2017/11/pexels-photo-296888.jpg12802560Datalumenhttps://www.datalumen.eu/wp-content/uploads/2019/06/datalumen_color_V1.1-340-156B.pngDatalumen2017-10-20 11:40:182017-11-22 14:16:18HOW TO REALLY SHIFT FROM IT-DRIVEN TO SELF-SERVICE ANALYTICS WITH DATA VIRTUALIZATION? LOOK BEYOND THE SHOP WINDOW.
Your privacy is important & we are committed to being transparent. By continuing to browse the site, you are agreeing to our use of cookies. To provide the best experiences, we use technologies like cookies to store and/or access device information.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Datalumen communication contains research, analysis or marketing information. You can always unsubribe and correction of your contact details may be made by using the Contact us link. Our Privacy Policy is available online.
window.cfields = {"1":"mobile_phone","5":"gdpr_consent"};
window._show_thank_you = function(id, message, trackcmp_url, email) {
var form = document.getElementById('_form_' + id + '_'), thank_you = form.querySelector('._form-thank-you');
form.querySelector('._form-content').style.display = 'none';
thank_you.innerHTML = message;
thank_you.style.display = 'block';
const vgoAlias = typeof visitorGlobalObjectAlias === 'undefined' ? 'vgo' : visitorGlobalObjectAlias;
var visitorObject = window[vgoAlias];
if (email && typeof visitorObject !== 'undefined') {
visitorObject('setEmail', email);
visitorObject('update');
} else if (typeof(trackcmp_url) != 'undefined' && trackcmp_url) {
// Site tracking URL to use after inline form submission.
_load_script(trackcmp_url);
}
if (typeof window._form_callback !== 'undefined') window._form_callback(id);
};
window._show_unsubscribe = function(id, message, trackcmp_url, email) {
var form = document.getElementById('_form_' + id + '_'), unsub = form.querySelector('._form-thank-you');
var branding = form.querySelector('._form-branding');
if (branding) {
branding.style.display = 'none';
}
form.querySelector('._form-content').style.display = 'none';
unsub.style.display = 'block';
form.insertAdjacentHTML('afterend', message)
const vgoAlias = typeof visitorGlobalObjectAlias === 'undefined' ? 'vgo' : visitorGlobalObjectAlias;
var visitorObject = window[vgoAlias];
if (email && typeof visitorObject !== 'undefined') {
visitorObject('setEmail', email);
visitorObject('update');
} else if (typeof(trackcmp_url) != 'undefined' && trackcmp_url) {
// Site tracking URL to use after inline form submission.
_load_script(trackcmp_url);
}
if (typeof window._form_callback !== 'undefined') window._form_callback(id);
};
window._show_error = function(id, message, html) {
var form = document.getElementById('_form_' + id + '_'),
err = document.createElement('div'),
button = form.querySelector('button'),
old_error = form.querySelector('._form_error');
if (old_error) old_error.parentNode.removeChild(old_error);
err.innerHTML = message;
err.className = '_error-inner _form_error _no_arrow';
var wrapper = document.createElement('div');
wrapper.className = '_form-inner';
wrapper.appendChild(err);
button.parentNode.insertBefore(wrapper, button);
var submitButton = form.querySelector('[id^="_form"][id$="_submit"]');
submitButton.disabled = false;
submitButton.classList.remove('processing');
if (html) {
var div = document.createElement('div');
div.className = '_error-html';
div.innerHTML = html;
err.appendChild(div);
}
};
window._show_pc_confirmation = function(id, header, detail, show, email) {
var form = document.getElementById('_form_' + id + '_'), pc_confirmation = form.querySelector('._form-pc-confirmation');
if (pc_confirmation.style.display === 'none') {
form.querySelector('._form-content').style.display = 'none';
pc_confirmation.innerHTML = "
var hideButton = document.getElementById('hideButton');
// Add event listener to the button
hideButton.addEventListener('click', function() {
var submitButton = document.querySelector('#_form_15_submit');
submitButton.disabled = false;
submitButton.classList.remove('processing');
var mp = document.querySelector('input[name="mp"]');
mp.value = '1';
window.location.href = window.location.href;
});
const vgoAlias = typeof visitorGlobalObjectAlias === 'undefined' ? 'vgo' : visitorGlobalObjectAlias;
var visitorObject = window[vgoAlias];
if (email && typeof visitorObject !== 'undefined') {
visitorObject('setEmail', email);
visitorObject('update');
} else if (typeof(trackcmp_url) != 'undefined' && trackcmp_url) {
// Site tracking URL to use after inline form submission.
_load_script(trackcmp_url);
}
if (typeof window._form_callback !== 'undefined') window._form_callback(id);
};
window._load_script = function(url, callback, isSubmit) {
var head = document.querySelector('head'), script = document.createElement('script'), r = false;
var submitButton = document.querySelector('#_form_15_submit');
script.charset = 'utf-8';
script.src = url;
if (callback) {
script.onload = script.onreadystatechange = function() {
if (!r && (!this.readyState || this.readyState == 'complete')) {
r = true;
callback();
}
};
}
script.onerror = function() {
if (isSubmit) {
if (script.src.length > 10000) {
_show_error("15", "Sorry, your submission failed. Please shorten your responses and try again.");
} else {
_show_error("15", "Sorry, your submission failed. Please try again.");
}
submitButton.disabled = false;
submitButton.classList.remove('processing');
}
}
head.appendChild(script);
};
(function() {
if (window.location.search.search("excludeform") !== -1) return false;
var getCookie = function(name) {
var match = document.cookie.match(new RegExp('(^|; )' + name + '=([^;]+)'));
return match ? match[2] : null;
}
var setCookie = function(name, value) {
var now = new Date();
var time = now.getTime();
var expireTime = time + 1000 * 60 * 60 * 24 * 365;
now.setTime(expireTime);
document.cookie = name + '=' + value + '; expires=' + now + ';path=/; Secure; SameSite=Lax;';
}
var addEvent = function(element, event, func) {
if (element.addEventListener) {
element.addEventListener(event, func);
} else {
var oldFunc = element['on' + event];
element['on' + event] = function() {
oldFunc.apply(this, arguments);
func.apply(this, arguments);
};
}
}
var _removed = false;
var form_to_submit = document.getElementById('_form_15_');
var allInputs = form_to_submit.querySelectorAll('input, select, textarea'), tooltips = [], submitted = false;
var getUrlParam = function(name) {
if (name.toLowerCase() !== 'email') {
var params = new URLSearchParams(window.location.search);
return params.get(name) || false;
}
// email is a special case because a plus is valid in the email address
var qString = window.location.search;
if (!qString) {
return false;
}
var parameters = qString.substr(1).split('&');
for (var i = 0; i < parameters.length; i++) {
var parameter = parameters[i].split('=');
if (parameter[0].toLowerCase() === 'email') {
return parameter[1] === undefined ? true : decodeURIComponent(parameter[1]);
}
}
return false;
};
var acctDateFormat = "%d/%m/%Y";
var getNormalizedDate = function(date, acctFormat) {
var decodedDate = decodeURIComponent(date);
if (acctFormat && acctFormat.match(/(%d|%e).*%m/gi) !== null) {
return decodedDate.replace(/(d{2}).*(d{2}).*(d{4})/g, '$3-$2-$1');
} else if (Date.parse(decodedDate)) {
var dateObj = new Date(decodedDate);
var year = dateObj.getFullYear();
var month = dateObj.getMonth() + 1;
var day = dateObj.getDate();
return `${year}-${month < 10 ? `0${month}` : month}-${day < 10 ? `0${day}` : day}`;
}
return false;
};
var getNormalizedTime = function(time) {
var hour, minutes;
var decodedTime = decodeURIComponent(time);
var timeParts = Array.from(decodedTime.matchAll(/(d{1,2}):(d{1,2})W*([AaPp][Mm])?/gm))[0];
if (timeParts[3]) { // 12 hour format
var isPM = timeParts[3].toLowerCase() === 'pm';
if (isPM) {
hour = parseInt(timeParts[1]) === 12 ? '12' : `${parseInt(timeParts[1]) + 12}`;
} else {
hour = parseInt(timeParts[1]) === 12 ? '0' : timeParts[1];
}
} else { // 24 hour format
hour = timeParts[1];
}
var normalizedHour = parseInt(hour) < 10 ? `0${parseInt(hour)}` : hour;
var minutes = timeParts[2];
return `${normalizedHour}:${minutes}`;
};
for (var i = 0; i < allInputs.length; i++) {
var regexStr = "field[(d+)]";
var results = new RegExp(regexStr).exec(allInputs[i].name);
if (results != undefined) {
allInputs[i].dataset.name = allInputs[i].name.match(/[time]$/)
? `${window.cfields[results[1]]}_time`
: window.cfields[results[1]];
} else {
allInputs[i].dataset.name = allInputs[i].name;
}
var fieldVal = getUrlParam(allInputs[i].dataset.name);
if (fieldVal) {
if (allInputs[i].dataset.autofill === "false") {
continue;
}
if (allInputs[i].type == "radio" || allInputs[i].type == "checkbox") {
if (allInputs[i].value == fieldVal) {
allInputs[i].checked = true;
}
} else if (allInputs[i].type == "date") {
allInputs[i].value = getNormalizedDate(fieldVal, acctDateFormat);
} else if (allInputs[i].type == "time") {
allInputs[i].value = getNormalizedTime(fieldVal);
} else {
allInputs[i].value = fieldVal;
}
}
}
var remove_tooltips = function() {
for (var i = 0; i < tooltips.length; i++) {
tooltips[i].tip.parentNode.removeChild(tooltips[i].tip);
}
tooltips = [];
};
var remove_tooltip = function(elem) {
for (var i = 0; i < tooltips.length; i++) {
if (tooltips[i].elem === elem) {
tooltips[i].tip.parentNode.removeChild(tooltips[i].tip);
tooltips.splice(i, 1);
return;
}
}
};
var create_tooltip = function(elem, text) {
var tooltip = document.createElement('div'),
arrow = document.createElement('div'),
inner = document.createElement('div'), new_tooltip = {};
if (elem.type != 'radio' && elem.type != 'checkbox') {
tooltip.className = '_error';
arrow.className = '_error-arrow';
inner.className = '_error-inner';
inner.innerHTML = text;
tooltip.appendChild(arrow);
tooltip.appendChild(inner);
elem.parentNode.appendChild(tooltip);
} else {
tooltip.className = '_error-inner _no_arrow';
tooltip.innerHTML = text;
elem.parentNode.insertBefore(tooltip, elem);
new_tooltip.no_arrow = true;
}
new_tooltip.tip = tooltip;
new_tooltip.elem = elem;
tooltips.push(new_tooltip);
return new_tooltip;
};
var resize_tooltip = function(tooltip) {
var rect = tooltip.elem.getBoundingClientRect();
var doc = document.documentElement,
scrollPosition = rect.top - ((window.pageYOffset || doc.scrollTop) - (doc.clientTop || 0));
if (scrollPosition < 40) {
tooltip.tip.className = tooltip.tip.className.replace(/ ?(_above|_below) ?/g, '') + ' _below';
} else {
tooltip.tip.className = tooltip.tip.className.replace(/ ?(_above|_below) ?/g, '') + ' _above';
}
};
var resize_tooltips = function() {
if (_removed) return;
for (var i = 0; i < tooltips.length; i++) {
if (!tooltips[i].no_arrow) resize_tooltip(tooltips[i]);
}
};
var validate_field = function(elem, remove) {
var tooltip = null, value = elem.value, no_error = true;
remove ? remove_tooltip(elem) : false;
if (elem.type != 'checkbox') elem.className = elem.className.replace(/ ?_has_error ?/g, '');
if (elem.getAttribute('required') !== null) {
if (elem.type == 'radio' || (elem.type == 'checkbox' && /any/.test(elem.className))) {
var elems = form_to_submit.elements[elem.name];
if (!(elems instanceof NodeList || elems instanceof HTMLCollection) || elems.length <= 1) {
no_error = elem.checked;
}
else {
no_error = false;
for (var i = 0; i < elems.length; i++) {
if (elems[i].checked) no_error = true;
}
}
if (!no_error) {
tooltip = create_tooltip(elem, "Please select an option.");
}
} else if (elem.type =='checkbox') {
var elems = form_to_submit.elements[elem.name], found = false, err = [];
no_error = true;
for (var i = 0; i < elems.length; i++) {
if (elems[i].getAttribute('required') === null) continue;
if (!found && elems[i] !== elem) return true;
found = true;
elems[i].className = elems[i].className.replace(/ ?_has_error ?/g, '');
if (!elems[i].checked) {
no_error = false;
elems[i].className = elems[i].className + ' _has_error';
err.push("Checking %s is required".replace("%s", elems[i].value));
}
}
if (!no_error) {
tooltip = create_tooltip(elem, err.join(' '));
}
} else if (elem.tagName == 'SELECT') {
var selected = true;
if (elem.multiple) {
selected = false;
for (var i = 0; i < elem.options.length; i++) {
if (elem.options[i].selected) {
selected = true;
break;
}
}
} else {
for (var i = 0; i < elem.options.length; i++) {
if (elem.options[i].selected
&& (!elem.options[i].value
|| (elem.options[i].value.match(/n/g)))
) {
selected = false;
}
}
}
if (!selected) {
elem.className = elem.className + ' _has_error';
no_error = false;
tooltip = create_tooltip(elem, "Please select an option.");
}
} else if (value === undefined || value === null || value === '') {
elem.className = elem.className + ' _has_error';
no_error = false;
tooltip = create_tooltip(elem, "This field is required.");
}
}
if (no_error && (elem.id == 'field[]' || elem.id == 'ca[11][v]')) {
if (elem.className.includes('phone-input-error')) {
elem.className = elem.className + ' _has_error';
no_error = false;
}
}
if (no_error && elem.name == 'email') {
if (!value.match(/^[+_a-z0-9-'&=]+(.[+_a-z0-9-']+)*@[a-z0-9-]+(.[a-z0-9-]+)*(.[a-z]{2,})$/i))
{
elem.className = elem.className + ' _has_error';
no_error = false;
tooltip = create_tooltip(elem, "Enter a valid email address.");
} else
{
// New code to reject hotmail.com, yahoo.com, gmail.com addresses and addresses containing the firstname field as the domain
var domain = value.split('@')[1].toLowerCase();
var firstname = document.getElementById('firstname').value.toLowerCase();
var lastname = document.getElementById('lastname').value.toLowerCase();
if (domain === 'hotmail' || domain === 'aol' || domain === 'gmail' || domain === 'yahoo' || domain === 'live' || domain === 'outlook' || domain === 'msn.com' || domain.includes(firstname) || domain.includes(lastname))
{
elem.className = elem.className + ' _has_error';
no_error = false;
tooltip = create_tooltip(elem, "We do not accept private email addresses. Please submit your valid business email.");
}
}
}
if (no_error && /date_field/.test(elem.className)) {
if (!value.match(/^dddd-dd-dd$/)) {
elem.className = elem.className + ' _has_error';
no_error = false;
tooltip = create_tooltip(elem, "Enter a valid date.");
}
}
tooltip ? resize_tooltip(tooltip) : false;
return no_error;
};
var needs_validate = function(el) {
if(el.getAttribute('required') !== null){
return true
}
if(el.name === 'email' && el.value !== ""){
return true
}
if((el.id == 'field[]' || el.id == 'ca[11][v]') && el.className.includes('phone-input-error')){
return true
}
return false
};
var validate_form = function(e) {
var err = form_to_submit.querySelector('._form_error'), no_error = true;
if (!submitted) {
submitted = true;
for (var i = 0, len = allInputs.length; i < len; i++) {
var input = allInputs[i];
if (needs_validate(input)) {
if (input.type == 'tel') {
addEvent(input, 'blur', function() {
this.value = this.value.trim();
validate_field(this, true);
});
}
if (input.type == 'text' || input.type == 'number' || input.type == 'time') {
addEvent(input, 'blur', function() {
this.value = this.value.trim();
validate_field(this, true);
});
addEvent(input, 'input', function() {
validate_field(this, true);
});
} else if (input.type == 'radio' || input.type == 'checkbox') {
(function(el) {
var radios = form_to_submit.elements[el.name];
for (var i = 0; i < radios.length; i++) {
addEvent(radios[i], 'click', function() {
validate_field(el, true);
});
}
})(input);
} else if (input.tagName == 'SELECT') {
addEvent(input, 'change', function() {
validate_field(this, true);
});
} else if (input.type == 'textarea'){
addEvent(input, 'input', function() {
validate_field(this, true);
});
}
}
}
}
remove_tooltips();
for (var i = 0, len = allInputs.length; i 31 && (charCode 57) && charCode !== 8) {
e.preventDefault();
}
});
};
var showPhoneInputError = function(inputId) {
var errorMessage = document.getElementById("error-msg-" + inputId);
var input = document.getElementById(inputId);
errorMessage.classList.add("phone-error");
errorMessage.classList.remove("phone-error-hidden");
input.classList.add("phone-input-error");
};
window['recaptcha_callback'] = function() {
// Get all recaptchas in the DOM (there may be more than one form on the page).
var recaptchas = document.getElementsByClassName("g-recaptcha");
for (var i in recaptchas) {
// Set the recaptcha element ID, so the recaptcha can be applied to each element.
var recaptcha_id = "recaptcha_" + i;
recaptchas[i].id = recaptcha_id;
var el = document.getElementById(recaptcha_id);
if (el != null) {
var sitekey = el.getAttribute("data-sitekey");
var stoken = el.getAttribute("data-stoken");
grecaptcha.render(recaptcha_id, {"sitekey":sitekey,"stoken":stoken});
}
}
}; _load_script(
"https://www.google.com/recaptcha/api.js?onload=recaptcha_callback&render=explicit"
);
var _form_serialize = function(form){if(!form||form.nodeName!=="FORM"){return }var i,j,q=[];for(i=0;i<form.elements.length;i++){if(form.elements[i].name===""){continue}switch(form.elements[i].nodeName){case"INPUT":switch(form.elements[i].type){case"tel":q.push(form.elements[i].name+"="+encodeURIComponent(form.elements[i].previousSibling.querySelector('div.iti__selected-dial-code').innerText)+encodeURIComponent(" ")+encodeURIComponent(form.elements[i].value));break;case"text":case"number":case"date":case"time":case"hidden":case"password":case"button":case"reset":case"submit":q.push(form.elements[i].name+"="+encodeURIComponent(form.elements[i].value));break;case"checkbox":case"radio":if(form.elements[i].checked){q.push(form.elements[i].name+"="+encodeURIComponent(form.elements[i].value))}break;case"file":break}break;case"TEXTAREA":q.push(form.elements[i].name+"="+encodeURIComponent(form.elements[i].value));break;case"SELECT":switch(form.elements[i].type){case"select-one":q.push(form.elements[i].name+"="+encodeURIComponent(form.elements[i].value));break;case"select-multiple":for(j=0;j {
if (key !== 'hideButton') {
formData.append(key, value);
}
//formData.append(key, value);
});
let request = {
headers: {
"Accept": "application/json"
},
body: formData,
method: "POST"
};
