TextPipe Lite, developed by DataMystic, is a specialized, entry-level data scrubbing and extraction tool designed to process, transform, and clean unstructured text or tabular data without code. It relies on a “pipeline” architecture where you pass text through a modular stack of built-in filters to fix formats, strip noise, and re-export files.
While TextPipe Pro handles enterprise automation via command-line scheduling and COM scripting, TextPipe Lite is built for desktop-driven, single-click automation of repetitive workflows. Core Data Cleaning Capabilities
TextPipe Lite automates several time-consuming scrubbing tasks using more than 100 manipulation filters:
Search and Replace: Handles multi-line replacements, case-insensitive transformations, and basic pattern matching. For example, it can standardize malformed text like “G.P.O. Box”, “gpo box”, and “GPO Box” into a single uniform format in one sweep.
Whitespace and Text Trimming: Automatically strips leading, trailing, or repeated blank spaces that corrupt database uploads.
Deduplication: Instantly sorts large text chunks and removes exact duplicate records.
Format Conversions: Translates text between systems, such as converting files from EBCDIC (mainframe) to ASCII (PC).
Data Extraction: Isolates and copies out matching data patterns, such as parsing structural tables to extract only email addresses or specific numeric columns. Step-by-Step: Automating a Data Cleaning Flow
To automate a recurring data cleaning task in TextPipe Lite, follow this workflow: 1. Define Your Input Source Open TextPipe Lite.
Drag and drop your messy text or CSV files into the workspace, or add a specific folder directory for batch processing. 2. Build the Filter Pipeline Navigate to the Filter Library on the left panel.
Select and stack filters sequentially to build your pipeline. For instance: Add “Convert Case” to normalize messy casing. Add “Remove Duplicate Lines” to prevent metric skewing.
Add “Find and Replace” using Pattern Matching to isolate and fix corrupt strings. 3. Test and Preview
Use the Trial Run or Debug option to process a small, harmless snippet of your data.
View the side-by-side comparison of raw text versus transformed text to ensure your rules do not overwrite critical fields. 4. Save and Reuse the Filter File How to Create Data-Cleaning Auto-Pipelines With Python
Leave a Reply