Text to Diagram using ChatGPT or CoPilot

From casual text to this diagram using ChatGPT:

Exhibit 1: Final Diagram

I recently created a whiteboard video explaining the Life Cycle of Deleted SharePoint Library Documents. After recording the video, I scripted a voiceover. The script was concise, describing the process in brief, action-oriented steps.

However, when I posted the final screenshot of the whiteboard video in my blog, it seemed busy and potentially overwhelming. While the video itself builds up the information gradually, someone seeing the diagram for the first time might be puzzled.

To enhance the blog post accompanying the video, I decided to include a simple diagram for those who might not have time to watch a 5-minute video. Although I'm experienced with UML diagrams, the thought of spending additional time on this project was less appealing than catching up on some much-needed sleep.

Taking a Shortcut with ChatGPT

I asked ChatGPT to generate a flowchart from the voiceover script using Mermaid.js, a JavaScript library that simplifies diagram and flowchart creation. You can view the voiceover script in the attached PDF:

Voice Over Text.pdf

Download PDF • 73KB

ChatGPT Prompt:  “Using these steps, prepare the process using the syntax for mermaid.js diagram” 

Now, what is mermaid.js?

Mermaid.js is a JavaScript library that offers an easy-to-use syntax for creating various types of diagrams and flowcharts.

Mermaid software is a tool for diagramming and charting. I used a free version available at https://mermaid.js.org/.

Initial Diagram Attempt

The first version ChatGPT produced captured all the actions but missed the roles of different actors like the User and SharePoint Site Admin:

Exhibit 2: First Diagram

Refining the Diagram

Following several prompts to include missing scenarios and roles, the diagram needed further adjustments to accurately reflect the deletion and preservation stages in SharePoint.

Here are a few prompts I used to refine the flowchart:

• Include actors in the diagram:
  

• Add missing actions where the user explicitly deletes files from the first stage recycle bin, and the admin deletes from the second stage.
  

• Correct nodes where admin actions were misrepresented as automatic processes.
  

Next prompt: I think you omitted two scenarios: That is when the user explicitly deletes the files from the first stage recycle bin and the SP Site Admin deletes items from the Second stage recycle bin. Those were in my script. It is either those actions or background processes. I am not seeing those actors/actions.

Next prompt: You forgot to add when SP Admin can restore deleted item from Preservation Hold Library to Library.

Next prompt: Why this node: J -->|Retention Period Ends| K[Move to Second-Stage Recycle Bin, Admin Action] goes directly to the library?

Now we are starting to get somewhere:

Next prompt: J -->|Retention Period Ends| K[Move to Second-Stage Recycle Bin, Admin Action] --> Admin action is for different node. It is the background job/processes/system that monitors every 7 days.

Next prompt: Modify the diagram and add the Decision shape and Yes as an outcome for the following: Exceeded Quota, Time lapsed (30 days) and Time Exceeded.

..... A FEW MOMENTS LATER.....

There were a few more prompts to polish it off to get to the final version. I made a few final modifications myself. You can find the resulting mermaid.js script at the end of this article.

Keep in mind that the final result is not a proper UML Diagram. This diagram serves as a casual pictorial to help visualize the lifecycle of deleted documents.

As I was preparing this post, I realized something: even though it is not a formal UML Diagram, the standards are so deeply ingrained in my mind that I subconsciously tried to bring it as close to those standards as possible without realizing it (traditions!).

Aside from a few initial mistakes and omissions by ChatGPT, I wonder if you would prefer the simplicity of the very first or third rendition of the diagram?

Considering the purpose of the diagram, it might not need all the decision shapes. Let me know!

Some reflections on the process

• Overall Experience: I found it surprisingly smooth.
  

• Performance of AI in Assisting: It did a great job overall. It missed capturing some "actors" and processes that were part of my script but did capture all the nodes, although some were just text (except for the background/system processes).
  

• Using Mermaid for the First Time: It was my first time using Mermaid... I would probably be better off editing the diagram using mermaid.js directly after all the information was plotted by ChatGPT. Even at the end of this exercise, I was able to tweak the scripts faster than going back to the prompt.
  

• Key Consideration: I did provide initial text with my ChatGPT prompt, and knowing all the processes I needed a diagram for made it super easy for me to identify what was missing. Keep in mind, while all actors and logical operators (ORs and ANDs) were in the text, it wasn't structured the way a typical business process description would be.

Why You Would Benefit From This Method

Imagine you have business processes that are text-only. You can quickly transform them into diagrams that provide visual representation, leading to better understanding and compliance.

I typically start with drawing on a napkin when I envision something, and even the scripts I fed to ChatGPT were created after my whiteboard diagramming. However, many places I consult have only written processes, and this method is a fantastic way to quickly add significant value by creating the visuals.

Additionally, you might consider using this method if:

• You are not well-versed in UML: If you have a diagram with tons of nodes, this is a good starting point. You are not starting from scratch.
  

• You are experienced in UML but dislike adjusting the nodes, catching the arrows, linking, and drawing.
  

• No strict adherence to “UML Proper” is required, and you are not obligated to use standard diagram tools specific to your organization.
  

• You have carpal tunnel syndrome
  

• You are easily frustrated with tools like Visio.
  

• You just want to throw something casual together quickly without the hassle.
  

• The sheer volume of diagrams you need to produce from text makes manual creation impractical.

Why You Might Not Use This Method

• If you must use MS Visio and need to provide editable MS Visio files as part of your tasks.

• If you don’t have the steps already outlined in writing, as I had with my voice-over script.
  

Regarding Mermaid's Licensing

A trial version of Mermaid is limited to saving only 5 diagrams, but there is a workaround. You can save each final diagram script in Notepad or another text editor of your choice. Then, you can simply copy the scripts back into the editor if you need to modify the diagram. The PRO version isn’t very expensive and offers features like Teams, multi-user editing, and version history, which are beneficial for professional environments.

Disclaimer: I am not affiliated with the Mermaid company in any way.

Final Flowchart Script

flowchart TD

    A["Item Deleted"] --> B["First-Stage Recycle Bin"]

    B -- 30 Days Retention, User Access --> C["User Restores Item"]

    B --> DQ{"Exceeded Quota?"}

    DQ -- Yes --> D["Purge Older Items by System"]

    B --> EQ{"30 Days Elapse?"}

    EQ -- Yes --> E["Move to Second-Stage Recycle Bin"]

    B --> UQ{"User Deletes Item?"}

    UQ -- Yes --> E

    E -- Up to 93 Days, Admin Access --> F["Site Admin Restores Item"]

    E --> GQ{"Exceed Storage Quota?"}

    GQ -- Yes --> G["Purge Older Items by System"]

    E --> TQ{"Time Exceeded?"}

    TQ -- Yes --> H["Permanently Deleted by System"]

    E --> AQ{"Admin Deletes Item?"}

    AQ -- Yes --> H

    H -- Contact Microsoft within 14 days --> I["Contact Microsoft for Restoration"]

    I -- Possible Site/Subsite Restoration --> J1["Restoration Completed"]

    A --> RA{"Retention Applied?"}

    RA -- Yes --> J["Move to Preservation Hold Library"]

    J --> RQ{"Retention Period Ended?"}

    RQ -- Yes --> K["Move to Second-Stage Recycle Bin"]

    K --> E

    J --> M["Admin Restores Item from Preservation Hold"]

    M --> L["Item in Library"]

    C --> L

    F --> L

    J1 --> L

    style B fill:#00C853,stroke:#333,stroke-width:2px

    style C fill:#C8E6C9,stroke:#333,stroke-width:4px

    style DQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style D fill:#bbf,stroke:#333,stroke-width:2px

    style EQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style E fill:#FF6D00,stroke:#333,stroke-width:2px

    style UQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style F fill:#C8E6C9,stroke:#333,stroke-width:4px

    style GQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style G fill:#bbf,stroke:#333,stroke-width:2px

    style TQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style H fill:#D50000,stroke:#333,stroke-width:2px

    style AQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style I fill:#2962FF,stroke:#333,stroke-width:4px,color:#FFFFFF

    style J1 fill:#C8E6C9,stroke:#333,stroke-width:4px

    style RA fill:#FFF9C4,stroke:#333,stroke-width:2px

    style J fill:#FFD600,stroke:#333,stroke-width:2px

    style RQ fill:#FFF9C4

    style K fill:#ccf,stroke:#333,stroke-width:2px

    style M fill:#C8E6C9,stroke:#333,stroke-width:4px

    style L fill:#BBDEFB,stroke:#333,stroke-width:4px

Behind the Scenes: The Lifecycle of Deleted SharePoint Documents

Deleting documents from a SharePoint Library isn't as final as you might think. Let's dive into what actually happens behind the scenes and how SharePoint manages deleted documents.

First-Stage Recycle Bin

Initially, when documents are deleted, they are moved to the First-Stage Recycle Bin. This is your "Recycle bin" that is readily accessible within SharePoint, allowing users to recover accidentally deleted items. Documents stay here for 30 days, but if your storage quota is exceeded, SharePoint might purge the oldest items to make room for new deletions.

If necessary, you can restore these items back to the Library at any point during this 30-day window.

Second-Stage Recycle Bin

If items are deleted from the First-Stage Recycle Bin or if they remain there for more than 30 days, they automatically transition to the Second-Stage Recycle Bin.

Unlike the First-Stage, this bin is not accessible to regular users; only SharePoint Site Admins have this capability. Admins can recover these items and restore them to the Library on behalf of users.

Items in the Second-Stage Recycle Bin are held for up to 93 days. However, just like in the First-Stage, if the storage quota is exceeded, older items will be deleted permanently to free up space.

Permanent Deletion

If items are explicitly deleted from the Second-Stage Recycle Bin by an admin or if they've outstayed their welcome (exceeding the 93-day limit), they are permanently deleted from your Tenant. This action is irreversible.

However, you can contact Microsoft within 14 days to request a restoration, but they can only restore entire Sites or Subsites, not individual items.

Special Cases: Retention Policies

Now, what if your documents are under a retention policy? The process changes slightly.

When you delete a document under retention, it moves to the Preservation Hold Library instead of the typical Recycle Bins. Only Site Admins can access this library, and they can restore documents back to the Library if necessary.

Documents remain in the Preservation Hold Library until the end of their retention period. A weekly background job checks for items that have passed their retention period by more than 30 days, moving them directly to the Second-Stage Recycle Bin.

From this point, the process mirrors that of regular documents, as items in the Second-Stage Recycle Bin follow the same pathway towards final deletion.

Twist in the Tale: When M365 Classifiers Misfire on Compound Records

In records management, dealing with compound records—documents that are related and often must be retained together—can be complex. These documents pose a challenge when trying to apply appropriate retention policies. In this post, we’ll explore this dilemma, the limitations of auto-labeling and AI, and effective solutions, including using the Primary document method and Document Sets.

Before we dive into the dilemma of managing compound records, let’s understand what they are.

First, we have Institutional Records. These are standard records maintained for general administrative purposes and compliance with regulations. For example, meeting minutes, policy documents, and annual reports.

A good example would be Employee Records: personal information, job performance, and payroll records retained for a specified period according to regulations. You’d normally retain such records according to an established File Plan for each type of record.

Compound Records are groups of related documents that must be retained together due to their interdependencies. While each individual document might have different retention schedules by regulations, this may not be the case with compound records. They often support each other and are critical in legal disputes.

Examples:

• Adjudication Files: Documents related to a court case, such as evidence, witness statements, and legal arguments.

• Mortgage Documentation: Loan agreements, payment records, property deeds, and correspondence related to a mortgage.
  

Institutional records are your "normal" records, regularly managed to comply with regulatory requirements, while compound records require special attention due to their interconnected nature and potential legal implications.

The Dilemma of Compound Records

Consider the following scenario:

• Document A: Needs to be retained for 3 years.

• Document B: Needs to be retained for 7 years.
  

However, Document A supports Document B. If Document A is crucial for understanding Document B, then Document A cannot be deleted based on its content alone and should be retained for the same period as Document B - 7 years.

Note: The need for compound records does not justify non-compliance with regulatory requirements. But in special cases, such as a lawsuit, specific laws may supersede standard regulations, allowing for adjustments to retention schedules. These situations are evaluated case-by-case by corporate lawyers. I am here to explain the technical aspects of managing these records within Microsoft 365, not to provide legal advice.

Microsoft 365 provides several ways to automatically apply retention labels:

• Content Types: Retention labels based on predefined content categories.

• Sensitive Information Types: Retention labels for documents containing sensitive data.

• Keywords: Retention labels triggered by specific keywords in documents.

• Trainable Classifiers: AI models trained to recognize specific content within documents.

• Metadata: Retention labels applied using document metadata fields.

• Microsoft Syntex: Advanced AI for classifying and processing content.

• Event-Based Retention: Retention labels applied based on specific events.

But all these methods struggle with the relational context and dependencies between documents, making them less effective for managing compound records. AI lacks the ability to fully understand and manage complex dependencies without explicit metadata or predefined rules.

Some Solutions I'd consider

1. Primary Document Method

In the ideal world with properly designed information architecture, related documents should be stored in the same container, library, or folder. However, there are scenarios where this might not be the case due to various reasons such as company structure, security needs, or oversight. Additionally, when documents are in the same container, they may be mixed with unrelated documents.

In such situations, we can create a custom metadata field which will be used to track an ID of the Primary Document and ID of related documents. The end user can then enter the Document IDs for compound documents to establish relationships between the documents. We can then configure the Labels to apply to the supporting documents with the same retention schedule as the Primary Document.

We could also leverage the Power Automate Flow to aid the users in populating Document IDs using automation.

Steps to Use the Primary Document Method:

1.     Create Metadata Fields:

   Primary Document: "Related Documents" field

Secondary Documents: "Primary Document ID" field

2.     Configure Content Types:

Add crated metadata fields to the respective content types

3.     Manual Entry or Automation:  

Enter IDs of Primary Document for Related Documents

Enter Related Documents for Primary Document

Once the metadata fields are set, you need to apply retention labels based on the Primary Document.

You can leverage Power Automate Flow to ensure that the retention label applied to the Primary Document is also applied to all related documents.

Remember, that the users can manually override auto-applied labels when necessary. Note that some restrictions apply, such as the inability to override Regulatory Records, and some actions may require SharePoint Admin permissions.

The Primary document method works well for simple cases where one main document (the Primary) is supported by other related documents or when there is an absolute need not to have the documents in the same containers, for whatever reason.

Pros:

• Simple and easy to implement for single primary-secondary relationships.

• Ensures that all supporting documents are linked to the primary document.

• Solves the dilemma of linking dispersed documents.

Cons:

• May not work well for complex cases with multiple dependencies and interdependencies.

• Requires manual effort or automation to link documents.

• Prone to errors. Could be a “crutch” for bad Information Architecture.

2. Document Sets

The Primary document method, quite frankly, is not the best solution if you can group related files together. In such cases, using Document Sets is the best solution.

Document Sets allow you to group related documents and manage them as a single entity with unified metadata and retention policies.

At the same time, files within Document Sets can have their own unique metadata. Document Sets simplify retention and record management by applying retention to the entire container.

• If supporting documents are needed on an ad hoc basis, move them to the Legal Case Document Set from their original location.

• Ensure you preserve document metadata during this process, as it is required by court procedures.

• Use the "Move" function instead of "Copy" to transfer documents to the Document Set. This helps maintain metadata integrity.

This method is ideal for managing a set of documents instead of managing complex relationships between them.

Document Sets simplify compliance and records management.

In fact, Microsoft introduced Document Sets to address Legal Cases, among other things. To me, Document Sets are the best thing that happened to SharePoint since baked bread.

I am working on a comprehensive Document Sets tutorial, which I will publish shortly.

For now, I am just going to leave you with advice to investigate Document Sets if you haven’t already done so.

This post had an objective to bring to your attention compound files and some concerns when dealing with them.

And one more thing, I would like to make sure that when dealing with compound files you are aware of the following:

Potential Non-Compliance

• Retaining files beyond their designated retention period without a valid legal reason can breach data protection laws (e.g., Freedom of Information Act, Personal Health Information Protection Act).

• Storing personal information longer than necessary without justification can lead to legal liabilities.
  

Role of Lawyers and Records Managers

• Deciding when to transition a file to adjudication status requires careful consideration by legal counsel and records managers. It should be based on actual legal needs rather than speculative scenarios.