package uucp

  1. Overview
  2. Docs

Break properties.

These properties are mainly for the Unicode text segmentation and line breaking algorithm.

References.

Line break

type line = [
  1. | `AI
  2. | `AK
  3. | `AL
  4. | `AP
  5. | `AS
  6. | `B2
  7. | `BA
  8. | `BB
  9. | `BK
  10. | `CB
  11. | `CJ
  12. | `CL
  13. | `CM
  14. | `CP
  15. | `CR
  16. | `EX
  17. | `EB
  18. | `EM
  19. | `GL
  20. | `H2
  21. | `H3
  22. | `HL
  23. | `HY
  24. | `ID
  25. | `IN
  26. | `IS
  27. | `JL
  28. | `JT
  29. | `JV
  30. | `LF
  31. | `NL
  32. | `NS
  33. | `NU
  34. | `OP
  35. | `PO
  36. | `PR
  37. | `QU
  38. | `RI
  39. | `SA
  40. | `SG
  41. | `SP
  42. | `SY
  43. | `VF
  44. | `VI
  45. | `WJ
  46. | `XX
  47. | `ZW
  48. | `ZWJ
]

The type for line breaks.

val pp_line : Format.formatter -> line -> unit

pp_line ppf l prints an unspecified representation of l on ppf.

val line : Uchar.t -> line

line u is u's line break property.

Grapheme cluster break

type grapheme_cluster = [
  1. | `CN
  2. | `CR
  3. | `EX
  4. | `EB
  5. | `EBG
  6. | `EM
  7. | `GAZ
  8. | `L
  9. | `LF
  10. | `LV
  11. | `LVT
  12. | `PP
  13. | `RI
  14. | `SM
  15. | `T
  16. | `V
  17. | `XX
  18. | `ZWJ
]

The type for grapheme cluster breaks.

val pp_grapheme_cluster : Format.formatter -> grapheme_cluster -> unit

pp_grapheme_cluster ppf g prints an unspecified representation of g on ppf.

val grapheme_cluster : Uchar.t -> grapheme_cluster

grapheme_cluster u is u's grapheme cluster break property.

Word break

type word = [
  1. | `CR
  2. | `DQ
  3. | `EX
  4. | `EB
  5. | `EBG
  6. | `EM
  7. | `Extend
  8. | `FO
  9. | `GAZ
  10. | `HL
  11. | `KA
  12. | `LE
  13. | `LF
  14. | `MB
  15. | `ML
  16. | `MN
  17. | `NL
  18. | `NU
  19. | `RI
  20. | `SQ
  21. | `WSegSpace
  22. | `XX
  23. | `ZWJ
]

The type for word breaks.

val pp_word : Format.formatter -> word -> unit

pp_word ppf b prints an unspecified representation of b on ppf.

val word : Uchar.t -> word

word u is u's word break property.

Sentence break

type sentence = [
  1. | `AT
  2. | `CL
  3. | `CR
  4. | `EX
  5. | `FO
  6. | `LE
  7. | `LF
  8. | `LO
  9. | `NU
  10. | `SC
  11. | `SE
  12. | `SP
  13. | `ST
  14. | `UP
  15. | `XX
]

The type for sentence breaks.

val pp_sentence : Format.formatter -> sentence -> unit

pp_sentence ppf b prints an unspecified representation of b on ppf.

val sentence : Uchar.t -> sentence

sentence u is u's sentence break property.

Indic conjunct break

type indic_conjunct_break = [
  1. | `Consonant
  2. | `Extend
  3. | `Linker
  4. | `None
]

The type for Indic Conjunct Break.

val pp_indic_conjunct_break : Format.formatter -> indic_conjunct_break -> unit

pp_indic_conjunct_break ppf b prints an unspecified representation of b on ppf.

val indic_conjunct_break : Uchar.t -> indic_conjunct_break

indic_conjunct_break u is u's Indic conjunct break property.

East Asian width

type east_asian_width = [
  1. | `A
  2. | `F
  3. | `H
  4. | `N
  5. | `Na
  6. | `W
]

The type for East Asian widths.

val pp_east_asian_width : Format.formatter -> east_asian_width -> unit

pp_east_asian_width ppf w prints an unspecified representation of w on ppf.

val east_asian_width : Uchar.t -> east_asian_width

east_asian_width u is u's East Asian width property.

Terminal width

val tty_width_hint : Uchar.t -> int

tty_width_hint u approximates u's column width as rendered by a typical character terminal.

The current implementation of the function returns either 0, 1, 2 or -1. The value -1 is only returned for scalar values for which the property is non-sensical; clients are expected to sanitize their inputs and not to use the function with these scalar values which are those in range U+0001-U+001F (C0 controls without U+0000) and U+007F-U+009F (DELETE and C1 controls).

Note. Converting a string to normalization form C before folding this function over its scalar values will, in general, yield better approximations (e.g. on Hangul).

Warning. This is not a normative property and only a heuristic. If you find yourself using this function please read carefully the following lines.

This function is the moral equivalent of POSIX wcwidth, in that its purpose is to help align text displayed by a character terminal. It mimics wcwidth, as widely implemented, in yet another way: it is mostly wrong.

Computing column width is a surprisingly difficult task in general. Much of the software infrastructure still carries legacy assumptions about the nature of text harking back to the ASCII era. Different terminal emulators attempt to cope with general Unicode text in different ways, creating a fundamental problem: width of text fragments will vary across terminal emulators, with no way of getting feedback from the output layer back into the text-producing layer.

For example: on a modern Linux system, a collection of terminals will disagree on some or all of U+00AD, U+0CBF, and U+2029. They will likewise disagree about unassigned characters (category Cn), sometimes contradicting the system's wcwidth (e.g. U+0378, U+0530). Terminals using bare libxft will display complex scripts differently from terminals using HarfBuzz, and the rendering on OS X will be slightly different from both.

tty_width_hint uses a simple and predictable width algorithm, based on Markus Kuhn's portable wcwidth:

  • Scalar values in the ranges U+0001-U+001F (C0 controls without U+0000) and U+007F-U+009F (DELETE and C1 controls) have undefined width (-1).
  • Characters with East Asian Width Fullwidth or Wide have a width of 2.
  • Characters with General Category Mn, Me, Cf and U+0000 have a width of 0.
  • Most other characters have a width of 1, including Cn.

This approach works well, in that it gives results generally consistent with a wide range of terminals, for alphabetic scripts, and for east Asian syllabic and logographic scripts in non-decomposed form. Support varies for abjad scripts in the presence of vowel marks, and it mostly breaks down on abugidas.

Moreover, non-text symbols like Emoji or Yijing hexagrams will be incorrectly classified as 1-wide, but this in fact agrees with their rendering on many terminals.

Clients should not over-rely on tty_width_hint. It provides a best-effort approximation which will sometimes fail in practice.

Low level interface

module Low : sig ... end

Low level interface.

OCaml

Innovation. Community. Security.