Encoding is a set of rules used to represent data in a form that can be stored and transmitted to another process or system. Character encoding (e.g. Windows-1252, ISO-8859-1, UTF-8, UTF-16) refers to the way character data is represented as a series of bytes. Binary encoding (e.g. Base64) refers to the way binary data is transformed into a series of characters.
Questions tagged [encoding]
141 questions
45
votes
3 answers
Using PostgreSQL 8.4, how to convert bytea to text value in postgres?
In my application I insert data in database using C code, since the strings I receive from an untrusted source I have escaped them using PQescapeByteaConn of libpq library. Which is working perfectly fine i.e. results in Octet format String. See…
Amit
- 591
- 2
- 5
- 10
35
votes
1 answer
How do I set a SQL Server Unicode / NVARCHAR string to an emoji or Supplementary Character?
I want to set a Unicode string variable to particular character based on its Unicode code point.
I want to use a code point beyond 65535, but the SQL Server 2008 R2 database has a collation of SQL_Latin1_General_CP1_CI_AS.
According to Microsoft's…
Riley Major
- 1,965
- 4
- 19
- 37
26
votes
4 answers
SQLCMD command is not able to insert accents
I am trying to run sqlcmd.exe in order to setup a new database from command line. I am using SQL SERVER Express 2012 on Windows 7 64 bits.
Here's the command I use:
SQLCMD -S .\MSSQLSERVER08 -V 17 -E -i %~dp0\aqualogyDB.sql -o…
Oskytar
- 363
- 1
- 3
- 7
21
votes
5 answers
Why does the varchar datatype allow unicode values?
I have a table with a varchar column. It is allowing Trademark(™), copyright(©) and other Unicode characters as shown below.
Create table VarcharUnicodeCheck
(
col1 varchar(100)
)
insert into VarcharUnicodeCheck (col1) values ('MyCompany')
insert…
Shiva
- 807
- 3
- 8
- 17
20
votes
2 answers
Postgresql: set default psql client encoding
When I connect to a Postgresql DB using psql, I often get these messages:
=> SELECT * FROM question_view ;
ERROR: character with byte sequence 0xd7 0x9e in encoding "UTF8" has no equivalent in encoding "LATIN1"
Following this SO answer, I…
Adam Matan
- 12,079
- 30
- 82
- 96
20
votes
3 answers
PostgreSQL: difference between collations 'C' and 'C.UTF-8'
In PostgreSQL, what is the difference between collations C and C.UTF-8?
Both show up in rows of pg_collation. Is it perhaps the case that C.UTF-8 is the same as C with encoding UTF-8 regardless or what the actual encoding of a database is?
rookie099
- 368
- 1
- 2
- 9
19
votes
4 answers
SQL Server 2005/2008 UTF-8 Collation/Charset
I can't find option(s) directly to set UTF-8 rellated Collations/Charsets in SQL Server 2005/2008, same as is possible to set in another SQL engines, but in SQL Server 2005/2008 are there only Latin and SQL collations.
Is there some option to…
mKorbel
- 597
- 2
- 12
- 25
18
votes
1 answer
Postgres - Encoding, Collation and CType
List of databases
Name | Owner | Encoding | Collate | Ctype …
Rob P.
- 283
- 1
- 2
- 6
17
votes
4 answers
How To Strip Hebrew Accent Marks
I need a Char Encoding Trick to Strip Hebrew Accent Marks.
Sample Before
בְּרֵאשִׁ֖ית בָּרָ֣א אֱלֹהִ֑ים אֵ֥ת הַשָּׁמַ֖יִם וְאֵ֥ת הָאָֽרֶץ
Sample After
בראשית ברא אלהים את השמים ואת הארץ
Decrypted
- 273
- 1
- 6
17
votes
1 answer
Error starting SQL Server 2017 service. Error Code 3417
I have SQL Server 2017 installed on my computer. This is what SELECT @@VERSION returns:
Microsoft SQL Server 2017 (RTM-GDR) (KB4293803) - 14.0.2002.14 (X64) Jul 21 2018 07:47:45 Copyright (C) 2017 Microsoft Corporation Enterprise Edition…
Beginner
- 273
- 1
- 3
- 7
16
votes
1 answer
Encoding issue with SQL Server VARCHAR column retrieved in Python
We recently had an issue with encoding related to a field that's being stored as a varchar(120) in SQL Server. In SSMS, the varchar appears as:
"Who Killed JonBen‚t?"
However, when it's brought into python, it appears as:
I've researched this…
Eric
- 263
- 1
- 2
- 9
11
votes
2 answers
Find non-UTF-8 data in mysql
I have a MySQL database containing data that is input by our customers.
Customers are told their data should only be encoded as UTF-8, however this is currently not being enforced and some people are using characters not encoded in UTF-8.
We intend…
Bret
- 113
- 1
- 1
- 4
10
votes
1 answer
Convert MySQL database from latin1 to utf8mb4 - and take care of German umlauts
I am using 5.6.28 MySQL Community Server to host WordPress + ProPhoto blog in German language at CentOS 6.7 Linux server:
mysql> show tables;
+-----------------------+
| Tables_in_blog |
+-----------------------+
| wp_commentmeta |
|…
Alexander Farber
- 405
- 3
- 7
- 19
9
votes
3 answers
Is it possible to use OPENROWSET to import fixed width UTF8 encoded files?
I have an example data file with following contents and saved with UTF8 encoding.
oab~opqr
öab~öpqr
öab~öpqr
The format of this file is fixed width with columns 1 to 3 each being allocated 1 character and column 4 reserved 5 characters.
I have…
Martin Smith
- 87,941
- 15
- 255
- 354
9
votes
1 answer
MySQL: Illegal mix of collations
I've tried using a stored procedure to create indices, and got the following error:
ERROR 1267 (HY000): Illegal mix of collations (utf8_general_ci,IMPLICIT) and
(utf8_unicode_ci,IMPLICIT) for operation '='
There's no line…
Adam Matan
- 12,079
- 30
- 82
- 96