Currently the glance client can't operate on IPv6 address based
openstack controller IPs. The reason for this is the absence of
creation of a IPv6 socket in the glance client code (in https.py).
The glance client is trying to create sockets from the AF_INET
socket family but this will lead to errors when glance client makes
a call on the IPv6 IP addresses.
In order to fix this limitation, we ensure that if the hostname
resolves to IPv6 or an explicit IPv6 address is used to configure
the openstack controller - glance client shall be able to detect
that and then create a AF_INET6 socket family. In all other cases
a AF_INET socket is created. We default to IPv4 sockets in all
other cases.
Change-Id: I7d5a09675cd5dab2e39f0faeaa7c169291eedac6
Closes-bug: #1348030
This enables glanceclient to authenticate using Keystone v3
API and includes the addition of several new CLI arguments.
DocImpact
Change-Id: I863ba08d312363dc1ce4fc7822fb21ef53df1a4f
Setting stream=True with requests can lead to issues
with not closing the connection so the urllib3 connection
pool is not freed up, so only set stream=True if making
a request with application/octet-stream content-type.
See the body-content-workflow and keep-alive sections
in the requests docs here for more information:
http://docs.python-requests.org/en/latest/user/advanced/
Note that commit dbb242b changed the response body_iter
code to potentially return a six.StringIO object rather
than the old ResponseBodyIterator class and since the
images client code is not converting the body_iter into
a dict using json.loads, we have to do that directly
in the _request method where the body_iter is
constructed.
Co-authored-by: Flavio Percoco <fpercoco@redhat.com>
Change-Id: I86572b69c4511f933c9676108190271874346302
Partial-Bug: #1341777
This review implements blueprint python-request and replaces the old
http client implementation in favor of a new one based on
python-requests.
Major changes:
* raw_request and json_request removed since everything is now being
handled by the same method "_request"
* New methods that match HTTP's methods were added:
- get
- put
- post
- head
- patch
- delete
* Content-Type is now being "inferred" based on the data being sent:
- if it is file-like object it chunks the request
- if it is a python type not instance of basestring then it'll try
to serialize it to json
- Every other case will keep the incoming content-type and will send
the data as is.
* Glanceclient's HTTPSConnection implementation will be used if
no-compression flag is set to True.
Co-Author: Flavio Percoco<flaper87@gmail.com>
Change-Id: I09f70eee3e2777f52ce040296015d41649c2586a
F841 detects local variable is assigned to but never used.
This commit fixes the violations and enables F841 in gate.
Change-Id: Ic4dcac2733dfe334009327ac17aa3952cafaa63a
According to my diagnosis [1], currently v2 API client consumed a lot
of CPU times by JSON schema validation on image model initialization
stage for image listing case. Because each image entry will triggers
one validation operation which spent most CPU clocks. (My test env has
1002 image entries now, so it's O(N) logic obviously).
This fix changed the original logic, it doesn't validate each image
entry but do it only on first image entry for each page. This approach
is a trade-off, it balanced the data validity and performance.
As comparison [3], under the fixed new logic, the execution time of
image listing is about four times as fast as old logic.
[1]
Clock type: CPU
Ordered by: totaltime, desc
name ncall ttot
..nt/v2/images.py:34 Controller.list 1003 12.08480
../warlock/core.py:30 image.__init__ 1002 11.91074
..warlock/model.py:28 image.__init__ 1002 11.90463
..arlock/model.py:130 image.validate 1002 11.73838
..nschema/validators.py:388 validate 1002 11.73682
[2]
name ncall ttot
..nt/v2/images.py:35 Controller.list 1003 1.100710
..nceclient/v2/images.py:45 paginate 256.. 1.099965
../warlock/core.py:30 image.__init__ 1002 0.944762
..warlock/model.py:28 image.__init__ 1002 0.939725
..arlock/model.py:130 image.validate 51 0.779653
..nschema/validators.py:388 validate 51 0.779431
[3]
$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real 0m16.606s
user 0m10.629s
sys 0m2.540s
$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real 0m4.151s
user 0m1.080s
sys 0m0.092s
Change-Id: Ia6598d3c06a5ff516277053c2b6fa5db744fe9cf
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
To be able to create profiling traces for Glance, client should be
able to send special HTTP header that contains trace info.
This patch is as well important to be able to make cross project
traces. (Typical case nova calls glance via python client, if
profiler is initialized in nova, glance client will add extra
header, that will be parsed by special osprofiler middleware in glance
api)
Don't worry no security issue here, trace information is signed by HMAC
key that is setted in api-paste.ini. So only person that knows HMAC key
is able to send proper header.
Change-Id: Ib13084fbe9b33c2f3dee165f7d6c778546cce6ca
Passing mutable objects as default args is a known Python pitfall.
We'd better avoid this. This commit changes mutable default args with
None.
Change-Id: I78323383c8b6a9a516a2a217303124870822418b
If --os-cacert was passed to the cli the following error was
triggered: "cafile must be None or a byte string". This is
because 'load_verify_locations' requires a byte string to
be passed in.
We fix this by explicitly converting the argument to a byte
string.
We do this in 'VerifiedHTTPSConnection' rather than sooner, eg
during arg handling, as it will no longer be required should we
move to a different http library (eg requests).
Fixes bug 1301849.
Change-Id: I9014f5d040cae9f0b6f03d8f13de8419597560cb
... and update tests to match.
The missing slash results in a non-absolute DELETE request
being sent to the API.
E.g.
DELETE v2/images/62fac489-23b4-4929-87af-2e7236e8542b HTTP/1.1
This is not strictly valid http/1.1 - rfc2616 specifies that the path must
be absolute.
This doesn't cause a problem for the API server, but this can cause
problems if the API server is fronted by something else (see #133161).
It also means that the curl command logged in debug mode has a
bad url.
E.g.
curl -i -X DELETE ... http://10.0.0.13:9292v2/images/...
Change-Id: Ib0c749dedbfcf07303fcddae4512db61b0f3fd78
Closes-bug: #1327101
Currently when an image is updated, the purge property
header is only set to true in some cases, but when
required it isn't set to false
Change-Id: I885a82643d2620f393f21c36b3ad95cb7ed43f2c
Closes-Bug: 1318079
This debug line is causing tracebacks in the n-cpu logs for
tempest runs. Its because the logged data is sometimes unicode:
Traceback (most recent call last):
File "/usr/lib/python2.7/logging/__init__.py", line 846, in emit
msg = self.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 710, in format
return logging.StreamHandler.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
return fmt.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 674, in format
return logging.Formatter.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 467, in format
s = self._fmt % record.__dict__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1234: ordinal not in range(128)
Logged from file http.py, line 153
The change used correct encoding error handling policy for the log
which may includes non-ascii char.
Closes-bug: 1320655
Change-Id: I97f5f14b9beddcceb7fbd371062caf5a38a62a20
Currently glanceclient's v2 commands don't support modification
operations on an image's location attribute - the argparse specification
for the location attribute of the image-update command causes the image
id argument to be included in list of locations and so the command
parsing fails (because it causes the image id to appear to be missing).
Furthermore even if the 'locations' argument were to be accepted by
argparse (e.g. by changing the argument specs and using --id to specify
the image id) the command would still fail because the arguments are
passed directly to the schema which expects the value of the 'locations'
argument to be a valid dictionary (there is nobody to convert the
argument string to a python dictionary that the schema expects).
This commit adds the following location related commands to
glanceclient:
--location-add: Add a new location to the list of image locations.
--location-delete: Remove an existing location from the list of
image locations.
--location-update: Update the metadata of existing location.
The glanceclient.v2.images.Controller class has been agumented with
three new methods to support the commands listed above:
- add_location
- delete_locations
- update_location
The server has not been modified, i.e. all location related API requests
are passed to the server via HTTP PATCH requests and handled by the
server's image update function.
The v2 'image' and 'shell' related tests have also been supplemented.
Note that in order to use these options the server must be first
configured to expose location related info to the clients (i.e.
'show_multiple_locations' must be set to 'True").
I also added a mailmap entry for myself.
DocImpact
Closes-bug: #1271452
Co-Author: David Koo (koofoss) <david.koo@huawei.com>
Change-Id: Id1f320af05d9344645836359758e4aa227aafc69
Add the type to the parameters that require an integer
in the V1 shell to avoid sending an improper request.
Change-Id: Idb1ed39b11ca737fdd42d24e297c142f28dce35c
Class `Managers` from `glanceclient.common.base` module is similar to
class `apiclient:ManagerWithFind` from common code.
In this patch:
- class glanceclient.common.base:Managers replaced by
apiclient:ManagerWithFind
- module glanceclient.common.base marked as 'deprecated'
Related to bp common-client-library-2
Change-Id: I41da4a9188e97ca2c07b6234fc2ac0a877553d3f
This module is obsolete and has been dropped from oslo-incubator.
Use six instead (which is already done here).
Change-Id: I8e0b49cdfbddcf2d44a601ad06be2e75897905fc
Currently, calls (create, get, etc.) return only the Image to a
caller. In order to log a mapping the request IDs of both glanceclient
and the caller, the x-openstack-request-id header value is needed on
the server side. This change allows that value to be bubbled up and
returned to the caller so that the appropriate logging can occur.
The return_req_id parameter can be set by services that are logging
request IDs. Glance's request ID will then be returned via the
return_req_id parameter.
This is a prerequisite for Log Request ID Mappings nova-spec to
be completed; Change Ib9b820a0feeb0c0e828ed3e4fab8261f8761ba9a
Change I43be05c351f901cee5509c76cff6d69f060c0b3f is an example of
a caller using this.
Implements: blueprint return-req-id
Change-Id: Ia82aa14db5f0e453010514fffb9a25d7b0fc2fd1
Currently only download method supports --progress flag in v2 API.
This patch let upload method in v2 API support this flag too.
Change-Id: I1d22379c320adb47a2178697e546413b9257f987
Closes-Bug: #1286265
Switch to using network_utils for splitting the URL. The code
in oslo-incubator supports ipv6 urls
Change-Id: I76be6173b97eb000319d30b4e9232a5a7c4a5aba
Closes-Bug: #1298137
Head of oslo-incubator is commit id:
2fd457bf2ccbeb2b84ffb204778b6417cd5405ba
includes a fix for gettextutil.Message handling of deep copy
failures and adding a license header
Change-Id: Ie89de1f95bb6fb9d11058413e682c441c39524f1
This will help with porting to Python 3. List of merged commits:
04a1abe59ac39890f57d2eed57d3d1b12bb5d757
12bcdb71ffbe9ee1688beed1f0ddb0c198822682
2cfc1a78d8063cf20083cf7df796d730a576551c
302c7c80b503b8090e8118e45061e8903b5339a9
35dc1d797209f0b22ff3ed3b8a6059961563daee
3b248dd683d83fd564f740ea58565d1bc0bd6cc0
4246ce0f373aa8f8955a99a3b6288a32547d8e80
6650435004af149284f0badb2bb8c89005cc7dab
6d55e26aa379fd043c66d1b7c5bf59a1b4a54632
71c22e9b2fbf04fc91a7343d3de0e0c7f10e3988
84d461e859e4b1eb1a00b9b141df61c37da84d2d
8575d87af49ea276341908f83c8c51db13afca44
885828af2297b20a3f9f8e8f6647f7291f784743
8b2b0b743e84ceed7841cf470afed6a5da8e1d07
9f1e7eb4112067423f845bd99ec52256db1c8bb1
bd5dad97585208ea5e86d636f3dc3b669e361a41
be81d6ba70152e5cdd7e8e9d789b9733c6ff527f
bec3a5eb8157c0c552c6392ccf0da342ffb8da26
c178e567cb48d3bc67dcf06bcbb4c6a97df9f434
Closes-Bug: 1289690
Change-Id: I39fa5e3fa46ffe9448b381ed54759cb4270cc898
It is Python2-specific, and should not be used in code intended to be portable.
Change-Id: Ibebef1a7e51a7444538275d22d444c926b8b4dec
Closes-Bug: 1284677
Make help strings consistent to use "." at end of string.
Fix capitalization of API in one help string.
Change-Id: I7cc5289d881c5e58aad9c69b4668584cdeb0b376
In Python 3, __next__() has replaced next(). Also, call the next() function
rather than the next() method.
Closes-Bug: #1281866
Change-Id: I92b44508c9c875f16ad89ef8410d2c38092ab23d
Some backend stores e.g. RBD, will fail if told to create an
image without a valid size (RBD will fail to write to a zero-size
image). Here we add support to allow the image size to be provided
when doing an upload. The result is that the upload content-length
will be set if available either from checking the supplied file
object or as provided by user.
Closes-Bug: 1220197
Change-Id: Ia1f2ea5680a139750d931591949b3e0058148b4b
Jenkins