Examples of fixed-width encoding forms:
| Type |
Each character
encoded as |
Notes |
| 7-bit |
a single 7-bit quantity |
example: ISO 646 |
| 8-bit G0/G1 |
a single 8-bit quantity |
with constraints on use of C0 and C1 spaces |
| 8-bit |
a single 8-bit quantity |
with no constraints on use of C1 space |
| 8-bit EBCDIC |
a single 8-bit quantity |
with the EBCDIC conventions rather than ASCII conventions |
| 16-bit (UCS-2) |
a single 16-bit quantity |
within a code space of 0..FFFF |
| 32-bit (UCS-4) |
a single 32-bit quantity |
within a code space 0..7FFFFFFF |
| 32-bit (UTF-32) |
a single 32-bit quantity |
within a code space of 0..10FFFF |
| 16-bit DBCS process code |
a single 16-bit quantity |
example: UNIX widechar implementations of Asian CCS's |
| 32-bit DBCS process code |
a single 32-bit quantity |
example: UNIX widechar implementations of Asian CCS's |
|
DBCS Host |
two 8-bit quantities |
following IBM host conventions |
Examples of variable-width encoding forms:
| Name |
Characters are encoded as |
Notes |
|
UTF-8 |
a mix of one to four 8-bit code units in Unicode
and one to six code units in 10646 |
used only with Unicode/10646 |
|
UTF-16 |
a mix of one to two 16 bit code units |
used only with Unicode/10646 |
Boost中提供了一个UTF-8 Codecvt Facet,可以在utf8和UCS-4(Unicode-32)之间转换.
使用方式如下
//...
// My encoding type
typedef wchar_t ucs4_t;
std::locale old_locale;
std::locale utf8_locale(old_locale,new utf8_codecvt_facet<ucs4_t>);
// Set a New global locale
std::locale::global(utf8_locale);
// UCS-4 转换为 UTF-8
{
std::wofstream ofs("data.ucd");
ofs.imbue(utf8_locale);
std::copy(ucs4_data.begin(),ucs4_data.end(),
std::ostream_iterator<ucs4_t,ucs4_t>(ofs));
}
// 读入 UTF-8 ,转换为 UCS-4
std::vector<ucs4_t> from_file;
{
std::wifstream ifs("data.ucd");
ifs.imbue(utf8_locale);
ucs4_t item = 0;
while (ifs >> item) from_file.push_back(item);
}
//...
UTF-8 Codecvt Facet详见
http://www.boost.org/libs/serialization/doc/codecvt.html