WEEK 02 — DIGITAL TRANSFORMATION

Big Data, Porter's Value Chain, and Socio-Technical Systems

Exploring data structures, Big Data concepts, and organizational frameworks.

Dr. Davood WadiDigital Transformation
A heap of data — a table, a photo, an email, a video and a social post — with an arrow pointing to a red stack of coins

CHAPTER 01

Data Structures

CORE CONCEPT

Structured and Unstructured Data

Structured and unstructured data are two primary categories of data that differ in their organization, format, and ease of analysis.

Structured data refers to information that is organized in a predefined manner, typically in rows and columns, making it easily searchable and analyzable.

A table of rows and columns with a dark header row; a magnifying glass beside it finds one row, which is red

Examples:

  • Customer databases (e.g., names, addresses, phone numbers)
  • Transaction records (e.g., sales data)
  • Inventory lists
  • Sensor data in a structured format

Unstructured data refers to information that does not have a predefined format or structure.

A loose scatter with no rows or columns: a photo, a video, a web page, a social post, an email and a raw sensor signal

Examples:

  • Social media posts and comments
  • Emails and text messages
  • Images and videos
  • Web pages and blogs
  • Sensor data in raw format
Exercise · Sort into groups

A hotel chain holds all of the data below. Sort each item into structured or unstructured data.

      Tap a card, then tap the group it belongs to.

      CLASS DISCUSSION

      1. 1Why is it important to distinguish between Structured and Unstructured Data?
      2. 2How to distinguish between Structured and Unstructured Data?
      3. 3Why did companies focus more on Structured data vs. Unstructured data?
      Two people in profile face each other, each with a speech bubble, in conversation

      CHAPTER 02

      Big Data

      DEFINITION

      Big Data and the Three Vs

      Big Data refers to the vast volumes of structured and unstructured data that are generated at high velocity from various sources, including social media, sensors, devices, transactions, and more.

      Big Data is often characterized by the “Three Vs”:

      1Volume

      The amount of data generated is enormous, often measured in terabytes or petabytes.

      A person stands beside a stack of database discs nearly twice their height

      2Velocity

      Data is generated and processed at an unprecedented speed (e.g., Real-time).

      A stopwatch above a line of small tables racing, with speed lines behind each, into a screen showing a live signal

      3Variety

      Data comes in various formats, including structured data (like databases), and unstructured data (like text, images, and videos).

      Six formats side by side: a table, a photo, a video, an email, a web page and a social post
      Exercise · Match up

      A ride-hailing platform runs on Big Data. Match each V with the situation that shows it.

      V

      Situation

      Tap a term, then the example it goes with.

      CHAPTER 03

      Big Data Analytics Technology

      TECHNOLOGY

      Data Storage

      Data Lakes: Repositories that store vast amounts of raw data in its native format until it is needed for analysis.

      A pale blue lake in which raw data floats in its native formats: a photo, a video, an email, a table, a social post and a sensor signal

      Data Warehouses: Centralized repositories that store structured data from multiple sources, optimized for query and analysis.

      Three sources — a shop, a web page and a sensor signal — each send an arrow into a warehouse whose shelves hold identical tables in neat rows
      Knowledge Check

      A city transit agency wants to keep every raw camera clip, sensor log and passenger complaint in its original format, without deciding yet how it will be analysed. Where should it keep this data?

      TECHNOLOGY

      Data Processing Frameworks

      Batch Processing: Techniques that process large volumes of data at once, such as Apache Hadoop.

      A large block of twelve tables enters a gear all at once; an arrow leads from the gear to a screen showing a lightbulb, the insight

      Stream Processing: Techniques that analyze data in real-time as it is generated, such as Apache Kafka and Apache Flink.

      Single tables arrive one after another, with speed lines behind each, into the same gear; an arrow leads to the same screen showing a lightbulb, the insight

      TECHNOLOGY

      Analytics Tools

      Descriptive Analytics: Tools that summarize historical data to understand what has happened (e.g., dashboards, reporting tools).

      Predictive Analytics: Techniques that use statistical models and machine learning to forecast future outcomes based on historical data.

      Prescriptive Analytics: Advanced analytics that recommends actions based on data analysis, often using optimization algorithms.

      A timeline in three zones. Descriptive: a solid line of past data. Predictive: the line continues as a dashed forecast inside a widening pale blue band. Prescriptive: the path forks in two, and the red branch is the recommended actionDescriptivePredictivePrescriptive
      Exercise · Which one is it?

      A grocery chain analyses its sales data. Name the kind of analytics each case shows.

      A dashboard shows last month's sales for each store.

      Which concept does this case show?

      CHAPTER 04

      Porter's Value Chain

      FRAMEWORK

      1. The Porter Value Chain is a concept developed by Michael E. Porter in his 1985 book.
      2. It is a framework that helps businesses analyze their internal activities to identify areas where they can create value and gain a competitive advantage.
      3. The value chain breaks down a company's activities into primary and support activities.
      4. allowing organizations to understand how each contributes to the overall value delivered to customers.

      FRAMEWORK

      Primary and Support Activities

      Primary Activities

      • Inbound Logistics: Receiving, warehousing, and inventory management of raw materials.

      • Operations: Processes that transform inputs into the final product or service.

      • Outbound Logistics: Activities required to get the finished product to the customer, including storage and order fulfillment.

      • Marketing and Sales: Strategies and activities to promote and sell the product or service.

      • Service: Activities related to maintaining and enhancing the product's value after purchase.

      Support Activities

      • Procurement: The process of acquiring the goods and services a company needs to carry out its operations.

      • Technology Development: Activities related to managing and developing technology, including research and development, process automation, and product design.

      • Human Resource Management: Activities related to recruiting, hiring, training, and developing the workforce.

      • Firm Infrastructure: Organizational structure, management, planning, finance, and quality control systems that support the entire value chain.

      Exercise · Sort into groups

      A bicycle maker carries out the activities below. Sort each one into primary or support activities.

          Tap a card, then tap the group it belongs to.

          CHAPTER 05

          Data Value Chain

          FRAMEWORK

          Data Value Chain: Generation to Storage

          The Data Value Chain is a conceptual framework that outlines the various stages involved in transforming raw data into valuable insights and actionable outcomes.

          The data value chain as a column of eight circles joined by arrows, from data generation down to data utilization, with a feedback and iteration arrow running back up to the first. Lit in red: generation, collection, storageGenerationCollectionStorageProcessingAnalysisVisualizationInterpretationUtilizationFeedback and Iteration

          1. 1.Data Generation

            This is the initial stage where data is created or collected. Data can come from various sources, including sensors, transactions, social media, surveys, etc.

          2. 2.Data Collection

            In this stage, the generated data is gathered and stored. This may involve data entry (manual), data scraping, or automated data collection methods.

          3. 3.Data Storage

            Once collected, data needs to be stored in a way that allows for easy access and retrieval. This can involve databases, data lakes, or cloud storage solutions.

          FRAMEWORK

          Data Value Chain: Processing to Visualization

          The data value chain as a column of eight circles joined by arrows, from data generation down to data utilization, with a feedback and iteration arrow running back up to the first. Lit in red: processing, analysis, visualizationGenerationCollectionStorageProcessingAnalysisVisualizationInterpretationUtilizationFeedback and Iteration

          1. 4.Data Processing

            This stage involves cleaning, transforming, and organizing the data to make it suitable for analysis. Data processing may include removing duplicates, handling missing values, and converting data into a standardized format.

          2. 5.Data Analysis

            In this phase, analytical techniques are applied to the processed data to extract insights. This can involve statistical analysis, machine learning, data mining, and other analytical methods to identify patterns, trends, and correlations.

          3. 6.Data Visualization

            The insights gained from data analysis are often presented through visualizations, such as charts, graphs, and dashboards. This helps stakeholders understand the findings and make informed decisions.

          FRAMEWORK

          Data Value Chain: Interpretation to Feedback

          The data value chain as a column of eight circles joined by arrows, from data generation down to data utilization, with a feedback and iteration arrow running back up to the first. Lit in red: interpretation, utilization and the feedback loopGenerationCollectionStorageProcessingAnalysisVisualizationInterpretationUtilizationFeedback and Iteration

          1. 7.Data Interpretation

            This stage involves interpreting the visualized data and insights to derive meaningful conclusions. It requires domain knowledge and expertise to understand the implications of the data.

          2. 8.Data Utilization

            Finally, the insights are used to inform decision-making, drive strategy, improve processes, or create new products and services. This stage emphasizes the practical application of data insights to achieve business objectives.

          3. 9.Feedback and Iteration

            The Data Value Chain is often iterative, meaning that feedback from the utilization stage can lead to new data generation, collection, and analysis efforts. This continuous cycle helps organizations adapt and improve their data strategies over time.

          Exercise · Put it in order

          A coffee chain wants to know why some stores sell less in the afternoon. Put its work in the order of the Data Value Chain.

          Tap a step, then the number where it belongs.

          CHAPTER 06

          Socio-Technical Systems

          FRAMEWORK

          Socio-Technical Systems

          Socio-Technical Systems (STS) refer to an approach to complex organizational work design that recognizes the interaction between people (the social system) and technology (the technical system).

          The concept emphasizes that both social and technical elements must be considered together to optimize performance, enhance productivity, and improve overall system effectiveness.

          Two people (the social system) and a screen with a gear (the technical system), with arrows running both ways between them, all held inside one red frameSocial systemTechnical system

          THE SOCIAL SYSTEM

          Social System

          A social system refers to the network of relationships, interactions, and structures that exist among individuals and groups within a society or organization.

          It encompasses the social dynamics, norms, values, roles, and behaviors that shape how people relate to one another and work together.

          Six people joined by lines into a network: five in a ring, each linked to their neighbours and to one person in the middle

          Characteristics of a Social System

          Interdependence: The members of a social system are interdependent, meaning that the actions of one individual or group can impact others.

          Adaptability: Social systems can adapt to changes in their environment, including shifts in technology, market conditions, or social norms.

          Complexity: Social systems are complex and dynamic, with multiple interacting elements. This complexity can lead to emergent behaviors that are not easily predictable based on the individual components.

          Stability and Change: While social systems strive for stability and cohesion, they are also subject to change. Factors such as leadership changes, organizational restructuring, or shifts in external conditions can disrupt the status quo.

          THE TECHNICAL SYSTEM

          Technical System

          A technical system refers to the collection of tools, technologies, processes, and structures that are used to perform tasks and achieve objectives within an organization or a specific context.

          Components of a Technical System

          1. 1.Hardware

            This includes the physical devices and equipment used in the system, such as computers, servers, machinery, and other tools.

          2. 2.Software

            Software encompasses the programs and applications that run on hardware. This includes operating systems, productivity software, specialized applications, and databases that support various functions within the organization.

          3. 3.Processes

            Technical systems involve specific processes and workflows that dictate how tasks are performed. standard operating procedures, algorithms, and methodologies that guide the use of technology.

          4. 4.Data

            It includes the information that is processed, stored, and analyzed by the system.

          5. 5.Networks

            local area networks (LANs), wide area networks (WANs), the internet

          6. 6.User Interfaces

            The design of user interfaces (UIs) is crucial for ensuring that users can effectively interact with the technical system. Good UI design enhances usability and user experience, making it easier for individuals to perform their tasks.

          7. 7.Support and Maintenance

            troubleshooting, updates, training for users to keep the system running smoothly.

          Exercise · Sort into groups

          A hospital introduces a new patient-records system. Sort each element into the social system or the technical system.

              Tap a card, then tap the group it belongs to.

              CONCLUSION

              Summary

              Structured data is organized in rows and columns; unstructured data has no predefined format or structure.

              KEY TAKEAWAYS

              1. 1Big Data is characterized by Volume, Velocity, and Variety.
              2. 2Big data analytics technology stores, processes, and analyzes data to describe what has happened, forecast what will happen, and recommend actions.
              3. 3Porter's Value Chain breaks a company's activities into primary and support activities.
              4. 4The Data Value Chain transforms raw data into valuable insights and actionable outcomes.
              5. 5Socio-Technical Systems consider people and technology together.