Bayanat Labs

Arabic Text Normalizer

Apply a named, hash-identified normalisation policy to Arabic text and see every change: before and after, the change log, the lossy changes and the policy SHA-256.

Normalisation runs in your browser. Your text is not uploaded, stored or logged.

Text

Policy SHA-256

Lossy rules in this policy:

What is hashed?

The hash is the SHA-256 of the policy’s canonical JSON without its meta member: keys sorted, no whitespace, UTF-8.


          
        

      

    


    

      
Normalised text

      
      
      
      
      

        
        
      

      

        
Changes
0

        
Lossy changes
0

        
Diacritization ratio
0

        
Code points before → after
0 → 0

      

      

        
Lossy flags

        
    
        
    Informational flags
    
        
      
        
      
      
    
      
  
      

  
      
    
      
      
      Before and after
      
      
    
      
    
      
      changed
      lossy change
      removed
      ZWNJ invisible character
    
      
    
      
      
      Before
      
      
      After
      
    
      
    
  
      

  
      
    
      Change log
      
    
      One row per changed character, in the order the stages ran. Positions count code points in the original text.
      
    
      
      
      
        
      
          #
          Stage
          Rule
          Position in input
          Before
          After
          Lossy
        
      
        
      
      
    
      
    
  
      

  
      
    
      Compare with another policy
      
    
      
      
      
    
      
    
      
    
      
      
      
        
      
          Input
          
          
        
      
        
      
      
    
      
  
      

  
      
    
      How it is computed
      
    
        
      
      1. A policy sets one mode for each of the 18 normalisation stages; the stages always run in the same order.
        2. 
      
      2. Character data comes from the Unicode 18.0.0 database bundled with the library, so the output does not change from one browser to another.
        3. 
      
      3. Every change is logged with its position in the original text, and the offset map links each output character to its position in the input.
        4. 
      
      4. Changes that can make two different words identical, such as ⁧ى⁩ to ⁧ي⁩, are flagged as lossy.
        5. 
      
      5. The SHA-256 identifies the policy exactly: cite the policy id and its hash, and anyone can reproduce the same output.
        6. 
    
      
    
      This page runs the JavaScript library @bayanat-labs/arabic-normalize, which gives the same output as the Python library arabic-normalize on every conformance case.
      
  
      

  
      
    
      Need production-grade, rights-cleared Arabic data?
      
    
      Bayanat Labs builds licensed Arabic datasets with documented normalisation.
      
    See Bayanat Knowledge