Lifecycle in a Nutshell: From Documents to Records

Intro

Quick high-level look at the lifecycle from documents for beginners. Learn the basics in under 3 minutes!

Documents

Documents are digital or physical files created or received by an organization as part of its everyday business operations.

These can include emails, memos, reports, presentations, spreadsheets, or any other types of content.

Documents used for collaboration, communication, and sharing information. These are your working documents.

Records

At some point, the document is no longer needed for active use, such as when a project ends. This means the document will transition to a different stage as a record.

A Record Manager tags each document with retention labels, either manually or through automation.

The documents are then moved into a secure repository, where they are locked and stored for the duration of their retention period.

During this time, no one collaborates on these documents.

Disposition

After the retention period ends, records are destroyed in an End of Life event.

Some records undergo a disposition review before destruction.

Alternatively, records can be moved to archive storage after their retention period ends.

Final Note

Records are documents recognized for their lasting value and importance.

They serve as proof of past activities and must follow retention policies to meet legal, regulatory, and organizational requirements.

While some organizations may call all types of files "records" even when they are in the document stage, there is usually a difference between active, working documents and those formally declared as records.

Watch YouTube video:

Text to Diagram using ChatGPT or CoPilot

From casual text to this diagram using ChatGPT:

Exhibit 1: Final Diagram

I recently created a whiteboard video explaining the Life Cycle of Deleted SharePoint Library Documents. After recording the video, I scripted a voiceover. The script was concise, describing the process in brief, action-oriented steps.

However, when I posted the final screenshot of the whiteboard video in my blog, it seemed busy and potentially overwhelming. While the video itself builds up the information gradually, someone seeing the diagram for the first time might be puzzled.

To enhance the blog post accompanying the video, I decided to include a simple diagram for those who might not have time to watch a 5-minute video. Although I'm experienced with UML diagrams, the thought of spending additional time on this project was less appealing than catching up on some much-needed sleep.

Taking a Shortcut with ChatGPT

I asked ChatGPT to generate a flowchart from the voiceover script using Mermaid.js, a JavaScript library that simplifies diagram and flowchart creation. You can view the voiceover script in the attached PDF:

Voice Over Text

.pdf

Download PDF • 73KB

ChatGPT Prompt:  “Using these steps, prepare the process using the syntax for mermaid.js diagram” 

Now, what is mermaid.js?

Mermaid.js is a JavaScript library that offers an easy-to-use syntax for creating various types of diagrams and flowcharts.

Mermaid software is a tool for diagramming and charting. I used a free version available at https://mermaid.js.org/.

Initial Diagram Attempt

The first version ChatGPT produced captured all the actions but missed the roles of different actors like the User and SharePoint Site Admin:

Exhibit 2: First Diagram

Refining the Diagram

Following several prompts to include missing scenarios and roles, the diagram needed further adjustments to accurately reflect the deletion and preservation stages in SharePoint.

Here are a few prompts I used to refine the flowchart:

• Include actors in the diagram:
  

• Add missing actions where the user explicitly deletes files from the first stage recycle bin, and the admin deletes from the second stage.
  

• Correct nodes where admin actions were misrepresented as automatic processes.
  

Next prompt: I think you omitted two scenarios: That is when the user explicitly deletes the files from the first stage recycle bin and the SP Site Admin deletes items from the Second stage recycle bin. Those were in my script. It is either those actions or background processes. I am not seeing those actors/actions.

Next prompt: You forgot to add when SP Admin can restore deleted item from Preservation Hold Library to Library.

Next prompt: Why this node: J -->|Retention Period Ends| K[Move to Second-Stage Recycle Bin, Admin Action] goes directly to the library?

Now we are starting to get somewhere:

Next prompt: J -->|Retention Period Ends| K[Move to Second-Stage Recycle Bin, Admin Action] --> Admin action is for different node. It is the background job/processes/system that monitors every 7 days.

Next prompt: Modify the diagram and add the Decision shape and Yes as an outcome for the following: Exceeded Quota, Time lapsed (30 days) and Time Exceeded.

..... A FEW MOMENTS LATER.....

There were a few more prompts to polish it off to get to the final version. I made a few final modifications myself. You can find the resulting mermaid.js script at the end of this article.

Keep in mind that the final result is not a proper UML Diagram. This diagram serves as a casual pictorial to help visualize the lifecycle of deleted documents.

As I was preparing this post, I realized something: even though it is not a formal UML Diagram, the standards are so deeply ingrained in my mind that I subconsciously tried to bring it as close to those standards as possible without realizing it (traditions!).

Aside from a few initial mistakes and omissions by ChatGPT, I wonder if you would prefer the simplicity of the very first or third rendition of the diagram?

Considering the purpose of the diagram, it might not need all the decision shapes. Let me know!

Some reflections on the process

• Overall Experience: I found it surprisingly smooth.
  

• Performance of AI in Assisting: It did a great job overall. It missed capturing some "actors" and processes that were part of my script but did capture all the nodes, although some were just text (except for the background/system processes).
  

• Using Mermaid for the First Time: It was my first time using Mermaid... I would probably be better off editing the diagram using mermaid.js directly after all the information was plotted by ChatGPT. Even at the end of this exercise, I was able to tweak the scripts faster than going back to the prompt.
  

• Key Consideration: I did provide initial text with my ChatGPT prompt, and knowing all the processes I needed a diagram for made it super easy for me to identify what was missing. Keep in mind, while all actors and logical operators (ORs and ANDs) were in the text, it wasn't structured the way a typical business process description would be.

Why You Would Benefit From This Method

Imagine you have business processes that are text-only. You can quickly transform them into diagrams that provide visual representation, leading to better understanding and compliance.

I typically start with drawing on a napkin when I envision something, and even the scripts I fed to ChatGPT were created after my whiteboard diagramming. However, many places I consult have only written processes, and this method is a fantastic way to quickly add significant value by creating the visuals.

Additionally, you might consider using this method if:

• You are not well-versed in UML: If you have a diagram with tons of nodes, this is a good starting point. You are not starting from scratch.
  

• You are experienced in UML but dislike adjusting the nodes, catching the arrows, linking, and drawing.
  

• No strict adherence to “UML Proper” is required, and you are not obligated to use standard diagram tools specific to your organization.
  

• You have carpal tunnel syndrome
  

• You are easily frustrated with tools like Visio.
  

• You just want to throw something casual together quickly without the hassle.
  

• The sheer volume of diagrams you need to produce from text makes manual creation impractical.

Why You Might Not Use This Method

• If you must use MS Visio and need to provide editable MS Visio files as part of your tasks.

• If you don’t have the steps already outlined in writing, as I had with my voice-over script.
  

Regarding Mermaid's Licensing

A trial version of Mermaid is limited to saving only 5 diagrams, but there is a workaround. You can save each final diagram script in Notepad or another text editor of your choice. Then, you can simply copy the scripts back into the editor if you need to modify the diagram. The PRO version isn’t very expensive and offers features like Teams, multi-user editing, and version history, which are beneficial for professional environments.

Disclaimer: I am not affiliated with the Mermaid company in any way.

Final Flowchart Script

flowchart TD

    A["Item Deleted"] --> B["First-Stage Recycle Bin"]

    B -- 30 Days Retention, User Access --> C["User Restores Item"]

    B --> DQ{"Exceeded Quota?"}

    DQ -- Yes --> D["Purge Older Items by System"]

    B --> EQ{"30 Days Elapse?"}

    EQ -- Yes --> E["Move to Second-Stage Recycle Bin"]

    B --> UQ{"User Deletes Item?"}

    UQ -- Yes --> E

    E -- Up to 93 Days, Admin Access --> F["Site Admin Restores Item"]

    E --> GQ{"Exceed Storage Quota?"}

    GQ -- Yes --> G["Purge Older Items by System"]

    E --> TQ{"Time Exceeded?"}

    TQ -- Yes --> H["Permanently Deleted by System"]

    E --> AQ{"Admin Deletes Item?"}

    AQ -- Yes --> H

    H -- Contact Microsoft within 14 days --> I["Contact Microsoft for Restoration"]

    I -- Possible Site/Subsite Restoration --> J1["Restoration Completed"]

    A --> RA{"Retention Applied?"}

    RA -- Yes --> J["Move to Preservation Hold Library"]

    J --> RQ{"Retention Period Ended?"}

    RQ -- Yes --> K["Move to Second-Stage Recycle Bin"]

    K --> E

    J --> M["Admin Restores Item from Preservation Hold"]

    M --> L["Item in Library"]

    C --> L

    F --> L

    J1 --> L

    style B fill:#00C853,stroke:#333,stroke-width:2px

    style C fill:#C8E6C9,stroke:#333,stroke-width:4px

    style DQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style D fill:#bbf,stroke:#333,stroke-width:2px

    style EQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style E fill:#FF6D00,stroke:#333,stroke-width:2px

    style UQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style F fill:#C8E6C9,stroke:#333,stroke-width:4px

    style GQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style G fill:#bbf,stroke:#333,stroke-width:2px

    style TQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style H fill:#D50000,stroke:#333,stroke-width:2px

    style AQ fill:#FFF9C4,stroke:#333,stroke-width:2px

    style I fill:#2962FF,stroke:#333,stroke-width:4px,color:#FFFFFF

    style J1 fill:#C8E6C9,stroke:#333,stroke-width:4px

    style RA fill:#FFF9C4,stroke:#333,stroke-width:2px

    style J fill:#FFD600,stroke:#333,stroke-width:2px

    style RQ fill:#FFF9C4

    style K fill:#ccf,stroke:#333,stroke-width:2px

    style M fill:#C8E6C9,stroke:#333,stroke-width:4px

    style L fill:#BBDEFB,stroke:#333,stroke-width:4px

Behind the Scenes: The Lifecycle of Deleted SharePoint Documents

Deleting documents from a SharePoint Library isn't as final as you might think. Let's dive into what actually happens behind the scenes and how SharePoint manages deleted documents.

First-Stage Recycle Bin

Initially, when documents are deleted, they are moved to the First-Stage Recycle Bin. This is your "Recycle bin" that is readily accessible within SharePoint, allowing users to recover accidentally deleted items. Documents stay here for 30 days, but if your storage quota is exceeded, SharePoint might purge the oldest items to make room for new deletions.

If necessary, you can restore these items back to the Library at any point during this 30-day window.

Second-Stage Recycle Bin

If items are deleted from the First-Stage Recycle Bin or if they remain there for more than 30 days, they automatically transition to the Second-Stage Recycle Bin.

Unlike the First-Stage, this bin is not accessible to regular users; only SharePoint Site Admins have this capability. Admins can recover these items and restore them to the Library on behalf of users.

Items in the Second-Stage Recycle Bin are held for up to 93 days. However, just like in the First-Stage, if the storage quota is exceeded, older items will be deleted permanently to free up space.

Permanent Deletion

If items are explicitly deleted from the Second-Stage Recycle Bin by an admin or if they've outstayed their welcome (exceeding the 93-day limit), they are permanently deleted from your Tenant. This action is irreversible.

However, you can contact Microsoft within 14 days to request a restoration, but they can only restore entire Sites or Subsites, not individual items.

Special Cases: Retention Policies

Now, what if your documents are under a retention policy? The process changes slightly.

When you delete a document under retention, it moves to the Preservation Hold Library instead of the typical Recycle Bins. Only Site Admins can access this library, and they can restore documents back to the Library if necessary.

Documents remain in the Preservation Hold Library until the end of their retention period. A weekly background job checks for items that have passed their retention period by more than 30 days, moving them directly to the Second-Stage Recycle Bin.

From this point, the process mirrors that of regular documents, as items in the Second-Stage Recycle Bin follow the same pathway towards final deletion.