Similar to commit dec9c9f35 and log_curl_request,
this ignores decoding errors when logging response
headers by passing errors='ignore' to safe_encode.
Change-Id: Ic915a7d8334e9473f300c9db670a3a8f5cda8976
Closes-Bug: #1369756
In the case where v2 requests are sent to a server which is not running
head of tree which includes the v2 metadef code some 404 cases need to
be handled to enable standard requests to complete.
This patch aslo improves fetching schemas -- they are now only
fetched as needed.
Change-Id: I8c871f11b909337bd7df19b77e606772dbc634b2
Closes-bug: #1367326
API calls and shell commands added in this patch:
- CRUD for metadefs namespaces;
- CRUD for metadefs objects;
- CRUD for metadefs properites;
- CRD for metadefs resource types and resource type associations.
Change-Id: I6d15f749038e8fd24fc651f0b314df5be7c673ef
Implements: blueprint metadata-schema-catalog-support
Co-Authored-By: Facundo Maldonado <facundo.n.maldonado@intel.com>
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Co-Authored-By: Lakshmi N Sampath <lakshmi.sampath@hp.com>
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
The new version of requests (2.4.0) has updated underlying urllib3
to version 1.9. Unfortunately urllib3 introduced new exception
ProtocolError. Because of that unit tests in glance are failing:
ProtocolError: ('Connection aborted.', gaierror(-2, 'Name or service not known'))
To solve this problem new urllib3 exception is caught in the same place
that the old one was. Unfortunately both exception are still in use so
I couldn't remove the old one.
Change-Id: I55eef98e734c59b9b627f182768a633b2b701e43
Closes-Bug: #1364893
Python 3 changed the semantics of dict.items() [0], which now returns a
iterable 'view' instead of a list of tuples. This has the repercussion
that you can no longer check for membership of a key using:
key in dict.items()
This patch simply replaces that check with a test for the key existing
in the dict itself, rather than the items.
[0] http://legacy.python.org/dev/peps/pep-3106/
Closes-Bug: 1359880
Change-Id: I7c59b0432725b660c9fa7270cde2e07bf3ea77db
A bug was introduced which meant that the server SSL certificate was
not being verified. Here we make sure that it is checked (unless
the --insecure flag is used).
Helps guard against man-in-the-middle attack.
Change-Id: I08f30bf3906b6580c871729311343fa8eefda91b
Closes-bug: #1357430
To help end user uses profiling by glanceclient CLI directly, the
change added '--profile <HMAC_KEY>' argument to the shell.
This change also fixed a function regression, it was used to pass
necessary trace info to glance server via http header:
https://github.com/openstack/python-glanceclient/commit/dbb242b776908ca50ed8557ebfe7cfcd879366c8#diff-740dd7d9149f46fe4c01ef0ab7eb3bfbL196
In addition:
1. If client app want to enable profiling feature for glanceclient
object, please see: http://paste.openstack.org/show/85722/
2. Afer adding more trace point into Client object, as fundamental
requirement of osprofiler, what we need to notice caller is providing
correct messaging notifier in the context code.
The relevant change for glance server is ready at:
I45a19f5eb5304c2b78a9e12cbc0744941a807304
DocImpact
Change-Id: If42b69b2695a5f88536ecbc2c1b3592d9dbd880a
Signed-off-by: Zhi Yan Liu <lzy.dev@gmail.com>
Some proxy or gateway softwares, e.g. jumpgate [0], use url-pattern
based approach to match which hanlding logic needs to be triggered for
particular service calling when it received a http(s) call as a
middleman. The change fixed an issue which caused glanceclient send out
the request to a dis-normal url, which contains duplicated "/".
The change removed a wrong and duplicated code snippet from curl logging
function as well.
[0] http://goo.gl/yt52X1
Change-Id: Ic8b3920e11d400771ead7f9c44b615f10b4a5cef
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Currently the glance client can't operate on IPv6 address based
openstack controller IPs. The reason for this is the absence of
creation of a IPv6 socket in the glance client code (in https.py).
The glance client is trying to create sockets from the AF_INET
socket family but this will lead to errors when glance client makes
a call on the IPv6 IP addresses.
In order to fix this limitation, we ensure that if the hostname
resolves to IPv6 or an explicit IPv6 address is used to configure
the openstack controller - glance client shall be able to detect
that and then create a AF_INET6 socket family. In all other cases
a AF_INET socket is created. We default to IPv4 sockets in all
other cases.
Change-Id: I7d5a09675cd5dab2e39f0faeaa7c169291eedac6
Closes-bug: #1348030
Setting stream=True with requests can lead to issues
with not closing the connection so the urllib3 connection
pool is not freed up, so only set stream=True if making
a request with application/octet-stream content-type.
See the body-content-workflow and keep-alive sections
in the requests docs here for more information:
http://docs.python-requests.org/en/latest/user/advanced/
Note that commit dbb242b changed the response body_iter
code to potentially return a six.StringIO object rather
than the old ResponseBodyIterator class and since the
images client code is not converting the body_iter into
a dict using json.loads, we have to do that directly
in the _request method where the body_iter is
constructed.
Co-authored-by: Flavio Percoco <fpercoco@redhat.com>
Change-Id: I86572b69c4511f933c9676108190271874346302
Partial-Bug: #1341777
This review implements blueprint python-request and replaces the old
http client implementation in favor of a new one based on
python-requests.
Major changes:
* raw_request and json_request removed since everything is now being
handled by the same method "_request"
* New methods that match HTTP's methods were added:
- get
- put
- post
- head
- patch
- delete
* Content-Type is now being "inferred" based on the data being sent:
- if it is file-like object it chunks the request
- if it is a python type not instance of basestring then it'll try
to serialize it to json
- Every other case will keep the incoming content-type and will send
the data as is.
* Glanceclient's HTTPSConnection implementation will be used if
no-compression flag is set to True.
Co-Author: Flavio Percoco<flaper87@gmail.com>
Change-Id: I09f70eee3e2777f52ce040296015d41649c2586a
To be able to create profiling traces for Glance, client should be
able to send special HTTP header that contains trace info.
This patch is as well important to be able to make cross project
traces. (Typical case nova calls glance via python client, if
profiler is initialized in nova, glance client will add extra
header, that will be parsed by special osprofiler middleware in glance
api)
Don't worry no security issue here, trace information is signed by HMAC
key that is setted in api-paste.ini. So only person that knows HMAC key
is able to send proper header.
Change-Id: Ib13084fbe9b33c2f3dee165f7d6c778546cce6ca
Passing mutable objects as default args is a known Python pitfall.
We'd better avoid this. This commit changes mutable default args with
None.
Change-Id: I78323383c8b6a9a516a2a217303124870822418b
If --os-cacert was passed to the cli the following error was
triggered: "cafile must be None or a byte string". This is
because 'load_verify_locations' requires a byte string to
be passed in.
We fix this by explicitly converting the argument to a byte
string.
We do this in 'VerifiedHTTPSConnection' rather than sooner, eg
during arg handling, as it will no longer be required should we
move to a different http library (eg requests).
Fixes bug 1301849.
Change-Id: I9014f5d040cae9f0b6f03d8f13de8419597560cb
This debug line is causing tracebacks in the n-cpu logs for
tempest runs. Its because the logged data is sometimes unicode:
Traceback (most recent call last):
File "/usr/lib/python2.7/logging/__init__.py", line 846, in emit
msg = self.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 710, in format
return logging.StreamHandler.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
return fmt.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 674, in format
return logging.Formatter.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 467, in format
s = self._fmt % record.__dict__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1234: ordinal not in range(128)
Logged from file http.py, line 153
The change used correct encoding error handling policy for the log
which may includes non-ascii char.
Closes-bug: 1320655
Change-Id: I97f5f14b9beddcceb7fbd371062caf5a38a62a20
Currently glanceclient's v2 commands don't support modification
operations on an image's location attribute - the argparse specification
for the location attribute of the image-update command causes the image
id argument to be included in list of locations and so the command
parsing fails (because it causes the image id to appear to be missing).
Furthermore even if the 'locations' argument were to be accepted by
argparse (e.g. by changing the argument specs and using --id to specify
the image id) the command would still fail because the arguments are
passed directly to the schema which expects the value of the 'locations'
argument to be a valid dictionary (there is nobody to convert the
argument string to a python dictionary that the schema expects).
This commit adds the following location related commands to
glanceclient:
--location-add: Add a new location to the list of image locations.
--location-delete: Remove an existing location from the list of
image locations.
--location-update: Update the metadata of existing location.
The glanceclient.v2.images.Controller class has been agumented with
three new methods to support the commands listed above:
- add_location
- delete_locations
- update_location
The server has not been modified, i.e. all location related API requests
are passed to the server via HTTP PATCH requests and handled by the
server's image update function.
The v2 'image' and 'shell' related tests have also been supplemented.
Note that in order to use these options the server must be first
configured to expose location related info to the clients (i.e.
'show_multiple_locations' must be set to 'True").
I also added a mailmap entry for myself.
DocImpact
Closes-bug: #1271452
Co-Author: David Koo (koofoss) <david.koo@huawei.com>
Change-Id: Id1f320af05d9344645836359758e4aa227aafc69
Class `Managers` from `glanceclient.common.base` module is similar to
class `apiclient:ManagerWithFind` from common code.
In this patch:
- class glanceclient.common.base:Managers replaced by
apiclient:ManagerWithFind
- module glanceclient.common.base marked as 'deprecated'
Related to bp common-client-library-2
Change-Id: I41da4a9188e97ca2c07b6234fc2ac0a877553d3f
Currently, calls (create, get, etc.) return only the Image to a
caller. In order to log a mapping the request IDs of both glanceclient
and the caller, the x-openstack-request-id header value is needed on
the server side. This change allows that value to be bubbled up and
returned to the caller so that the appropriate logging can occur.
The return_req_id parameter can be set by services that are logging
request IDs. Glance's request ID will then be returned via the
return_req_id parameter.
This is a prerequisite for Log Request ID Mappings nova-spec to
be completed; Change Ib9b820a0feeb0c0e828ed3e4fab8261f8761ba9a
Change I43be05c351f901cee5509c76cff6d69f060c0b3f is an example of
a caller using this.
Implements: blueprint return-req-id
Change-Id: Ia82aa14db5f0e453010514fffb9a25d7b0fc2fd1
Switch to using network_utils for splitting the URL. The code
in oslo-incubator supports ipv6 urls
Change-Id: I76be6173b97eb000319d30b4e9232a5a7c4a5aba
Closes-Bug: #1298137
It is Python2-specific, and should not be used in code intended to be portable.
Change-Id: Ibebef1a7e51a7444538275d22d444c926b8b4dec
Closes-Bug: 1284677
In Python 3, __next__() has replaced next(). Also, call the next() function
rather than the next() method.
Closes-Bug: #1281866
Change-Id: I92b44508c9c875f16ad89ef8410d2c38092ab23d
Some backend stores e.g. RBD, will fail if told to create an
image without a valid size (RBD will fail to write to a zero-size
image). Here we add support to allow the image size to be provided
when doing an upload. The result is that the upload content-length
will be set if available either from checking the supplied file
object or as provided by user.
Closes-Bug: 1220197
Change-Id: Ia1f2ea5680a139750d931591949b3e0058148b4b
Using common method 'bool_from_string' from oslo strutils to replace
utils.string_to_bool.
partially implements blueprint common-client-library-2
Change-Id: I23924db3000feadcfe823c6cc979ea9752a13fa9
When using the cli the Glance client wraps the endpoint in a 'strip
version' function. This means that endpoints of the following forms can
both be used:
https://region-x.images.example.com:443/v1https://region-x.images.example.com:443
When calling the client library directly (as Ceilometer does) however
only endpoints of the second form work. The cli and library should handle
the two cases consistently.
Addresses bug 1243276.
Change-Id: Ice7b581fee32540a7057ba47433a10166a3caed2
the client should not log a server 50x response at ERROR level.
It already throws an exception, so the caller can care about what
it wants to do with this. This error will show up regularly in
nova that succeed, and there is no way to disable this message,
which isn't an error from Nova's perspective without turning off
*all* the logs.
Set this to debug instead, because from a caller perspective that's
what you want out of this.
Change-Id: I6d0efb53d1e81adf309f7fa580ec5a8073a811c5
Those modules have been combined in python 3. Therefore,
implementing the six module helps to have support to both versions.
Change-Id: I164a0f19790ff066d16d0cf4f0daa6f1097c848e
Closes-Bug: #1267181
six.iteritems() replaces dictionary.iteritems() on Python 2 and
dictionary.items() on Python 3.
six.string_types replaces basestring() in Python 2 and str
in Python 3.
Change-Id: Ia18510d167df35caec83626718010228e2140bc0
* Fix "TypeError: not all arguments converted during string formatting"
* Add a UT case to cover the path where the bug is in
Change-Id: I91a137c5c3a9a3cc603804bef5eaea14ae281c08
Closes-Bug: #1265730
In the process of unification of the clients code we should
reuse common functionality from Oslo.
Related to blueprint common-client-library-2
Change-Id: I3d82621cced57ad7bc14a2a40e4bef7522f233da
Matt Riedemann