Decode input and encode output

Currently glanceclient doesn't support non-ASCII characters for images
names and properties (names and values as well). This patch introduces 2
functions (utils.py) that will help encoding and decoding strings in a
more "secure" way.

About the ensure_(str|unicode) functions:

    They both try to use first the encoding used in stdin (or python's
    default encoding if that's None) and fallback to utf-8 if those
    encodings fail to decode a given text.

About the changes in glanceclient:

    The major change is that all inputs will be decoded and will kept as
    such inside the client's functions and will then be encoded before
    being printed / sent out the client.

    There are other small changes, all related to encoding to str,
    around in order to avoid fails during some conversions. i.e: quoting
    url encoded parameters.

Fixes bug: 1061150

Change-Id: I5c3ea93a716edfe284d19f6291d4e36028f91eb2

tests/test_utils.py

@@ -50,3 +50,36 @@ class TestUtils(testtools.TestCase):
         self.assertEqual("1MB", utils.make_size_human_readable(1048576))
         self.assertEqual("1.4GB", utils.make_size_human_readable(1476395008))
         self.assertEqual("9.3MB", utils.make_size_human_readable(9761280))
+
+    def test_ensure_unicode(self):
+        ensure_unicode = utils.ensure_unicode
+        self.assertEqual(u'True', ensure_unicode(True))
+        self.assertEqual(u'ni\xf1o', ensure_unicode("ni\xc3\xb1o",
+                                                    incoming="utf-8"))
+        self.assertEqual(u"test", ensure_unicode("dGVzdA==",
+                                                 incoming='base64'))
+
+        self.assertEqual(u"strange", ensure_unicode('\x80strange',
+                                                    errors='ignore'))
+
+        self.assertEqual(u'\xc0', ensure_unicode('\xc0',
+                                                 incoming='iso-8859-1'))
+
+        # Forcing incoming to ascii so it falls back to utf-8
+        self.assertEqual(u'ni\xf1o', ensure_unicode('ni\xc3\xb1o',
+                                                    incoming='ascii'))
+
+    def test_ensure_str(self):
+        ensure_str = utils.ensure_str
+        self.assertEqual("True", ensure_str(True))
+        self.assertEqual("ni\xc3\xb1o", ensure_str(u'ni\xf1o',
+                                                   encoding="utf-8"))
+        self.assertEqual("dGVzdA==\n", ensure_str("test",
+                                                  encoding='base64'))
+        self.assertEqual('ni\xf1o', ensure_str("ni\xc3\xb1o",
+                                               encoding="iso-8859-1",
+                                               incoming="utf-8"))
+
+        # Forcing incoming to ascii so it falls back to utf-8
+        self.assertEqual('ni\xc3\xb1o', ensure_str('ni\xc3\xb1o',
+                                                   incoming='ascii'))