The oslo.utils function has exception_to_unicode that can
replace glance util function exception_to_str.
So we don't to have this exception_to_str function in glance
anymore.
Change-Id: I332bc55558087920fdd6ae2d822bece5166f5ba6
The check used to verify if there are any properties to remove in the V2
image-update command will always execute even if there are no properties to
remove due to the fact that it checks if remove_prop is not None, when it is
actually a list and not a None value.
Closes-Bug: #1475053
Change-Id: Ia36e945b880de3514c73073a392bdb7dde13cf84
Previously when listing images via v2, the first image of each batch
was validated against the v2 schema.
This could lead to unpredictable behaviour where a particular image
which failed the schema check may or may not be the first of a batch.
In some cases it also meant that operation by one user could affect the
ability of another other to list images.
Change-Id: I22974a3e3d9cbdd254099780752ae45ff2a557af
Closes-bug: 1477910
This commit enables new flake8 checks:
* E265 block comment should start with '# '
* H405 multi line docstring summary not separated with an empty line
* E123 closing bracket does not match indentation of opening bracket's line
* H238 old style class declaration, use new style (inherit from `object`)
* E128 continuation line under-indented for visual indent
and makes related changes in the code.
Change-Id: Ie993afc930f6b74d7a990bcaa9fc0e9f5ba1585c
The v2 Glance client implementation ignores the marker attribute if
it's passed to the client as part of the kwargs. Include support
for the attribute, as the v1 client supports it and it makes it
easier to work with the Glance client.
Change-Id: Ifaa59129bc4178212b890ea591673cb154e0f110
Closes-bug: 1465373
Right now all invalid tags are skipped when filtering for image list. We
should change this behaviour to raise an exception when invalid tag
value is provided.
Additionally remove misplaced `pass` from one of the tests.
Closes-Bug: 1433962
Change-Id: If5148608a70ee019697ea2dcb84e19a53222d596
This code enables new syntax for sorting output with multiple
keys and directions based on API Working group sorting
guidelines.
It's a client code to consume API modified in change
Ie4ccfefa0492a3ac94cc7e22201f2f2be5c1cdbb
Example:
glance --os-image-api-version 2 --sort name:desc,size:asc
Implements-blueprint: glance-sorting-enhancements
DocImpact
Depends-On: Ie4ccfefa0492a3ac94cc7e22201f2f2be5c1cdbb
Change-Id: I36a9fa9f0508fea1235de2ac3a0d6a093e1af635
This fixes the same problem with leaking file descriptors after the port
to requests I16a7b02f2b10e506e91719712cf34ef0aea1afc0 does, but for the
v2 api client.
The paginate function in the list method has been refactored from a
recursive generator to a non-recursive generator, which avoids issues
with garbage collection holding the socket open.
Change-Id: Iaa7a421e8c1acafb7b23bb3f55e50b9d2a9d030a
Closes-Bug: #1428797
As a new user I found navigating the documentation difficult. The
flow was a bit unclear and searches bring up old versions of API
references that aren't included in the current documentation.
This
- provides an introduction to the tools similar to other projects
- generates API references for the v1 and v2 client
- fixes some minor docstring issues
- adds doc/* to pep8 tests to check the conf.py
The API generation code is cribbed from python-novaclient
Change-Id: I65772127679d7afd5e7e48ca7872366b01382f21
The oslo.utils libraries are moving away from namespace packages.
This requires oslo.utils>=1.2.0
bp drop-namespace-packages
Change-Id: I803df61e91eabb96329d859aef6bea03530fb84f
In v2, we use the link header during paginations to know when there are
more images available in the server that could be returned in the same
request. This way, it's possible to iterate over the generator returned
by list and consume the images in the server.
However, it's currently not possible to tell glanceclient the exact
number of images we want, which basically means that it'll *always* go
through the whole list of images in the server unless the limit is
implemented by the consumer.
DocImpact
Change-Id: I9f65a40d4eafda6320e5c7d94d03fcd1bbc93e33
Closes-bug: #1415035
This module now lives in oslo.utils, so import it from there instead.
Co-Authored-By: Ian Cordasco <ian.cordasco@rackspace.com>
Change-Id: Ib35dc840992433542490670781badd9529ec8947
The move to the requests library (dbb242b) broke the progress bar during
downloading an image with --progress enabled, due to requests returning
a generator, which has no __len__ function. This patch adds an iterable
wrapper which provides the length, sourced from Content-Length headers.
Closes-Bug: #1384664
Change-Id: I48598a824396bc6e7cba69eb3ce32e7c8f30a18a
In the case where v2 requests are sent to a server which is not running
head of tree which includes the v2 metadef code some 404 cases need to
be handled to enable standard requests to complete.
This patch aslo improves fetching schemas -- they are now only
fetched as needed.
Change-Id: I8c871f11b909337bd7df19b77e606772dbc634b2
Closes-bug: #1367326
This review implements blueprint python-request and replaces the old
http client implementation in favor of a new one based on
python-requests.
Major changes:
* raw_request and json_request removed since everything is now being
handled by the same method "_request"
* New methods that match HTTP's methods were added:
- get
- put
- post
- head
- patch
- delete
* Content-Type is now being "inferred" based on the data being sent:
- if it is file-like object it chunks the request
- if it is a python type not instance of basestring then it'll try
to serialize it to json
- Every other case will keep the incoming content-type and will send
the data as is.
* Glanceclient's HTTPSConnection implementation will be used if
no-compression flag is set to True.
Co-Author: Flavio Percoco<flaper87@gmail.com>
Change-Id: I09f70eee3e2777f52ce040296015d41649c2586a
According to my diagnosis [1], currently v2 API client consumed a lot
of CPU times by JSON schema validation on image model initialization
stage for image listing case. Because each image entry will triggers
one validation operation which spent most CPU clocks. (My test env has
1002 image entries now, so it's O(N) logic obviously).
This fix changed the original logic, it doesn't validate each image
entry but do it only on first image entry for each page. This approach
is a trade-off, it balanced the data validity and performance.
As comparison [3], under the fixed new logic, the execution time of
image listing is about four times as fast as old logic.
[1]
Clock type: CPU
Ordered by: totaltime, desc
name ncall ttot
..nt/v2/images.py:34 Controller.list 1003 12.08480
../warlock/core.py:30 image.__init__ 1002 11.91074
..warlock/model.py:28 image.__init__ 1002 11.90463
..arlock/model.py:130 image.validate 1002 11.73838
..nschema/validators.py:388 validate 1002 11.73682
[2]
name ncall ttot
..nt/v2/images.py:35 Controller.list 1003 1.100710
..nceclient/v2/images.py:45 paginate 256.. 1.099965
../warlock/core.py:30 image.__init__ 1002 0.944762
..warlock/model.py:28 image.__init__ 1002 0.939725
..arlock/model.py:130 image.validate 51 0.779653
..nschema/validators.py:388 validate 51 0.779431
[3]
$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real 0m16.606s
user 0m10.629s
sys 0m2.540s
$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real 0m4.151s
user 0m1.080s
sys 0m0.092s
Change-Id: Ia6598d3c06a5ff516277053c2b6fa5db744fe9cf
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
... and update tests to match.
The missing slash results in a non-absolute DELETE request
being sent to the API.
E.g.
DELETE v2/images/62fac489-23b4-4929-87af-2e7236e8542b HTTP/1.1
This is not strictly valid http/1.1 - rfc2616 specifies that the path must
be absolute.
This doesn't cause a problem for the API server, but this can cause
problems if the API server is fronted by something else (see #133161).
It also means that the curl command logged in debug mode has a
bad url.
E.g.
curl -i -X DELETE ... http://10.0.0.13:9292v2/images/...
Change-Id: Ib0c749dedbfcf07303fcddae4512db61b0f3fd78
Closes-bug: #1327101
Currently glanceclient's v2 commands don't support modification
operations on an image's location attribute - the argparse specification
for the location attribute of the image-update command causes the image
id argument to be included in list of locations and so the command
parsing fails (because it causes the image id to appear to be missing).
Furthermore even if the 'locations' argument were to be accepted by
argparse (e.g. by changing the argument specs and using --id to specify
the image id) the command would still fail because the arguments are
passed directly to the schema which expects the value of the 'locations'
argument to be a valid dictionary (there is nobody to convert the
argument string to a python dictionary that the schema expects).
This commit adds the following location related commands to
glanceclient:
--location-add: Add a new location to the list of image locations.
--location-delete: Remove an existing location from the list of
image locations.
--location-update: Update the metadata of existing location.
The glanceclient.v2.images.Controller class has been agumented with
three new methods to support the commands listed above:
- add_location
- delete_locations
- update_location
The server has not been modified, i.e. all location related API requests
are passed to the server via HTTP PATCH requests and handled by the
server's image update function.
The v2 'image' and 'shell' related tests have also been supplemented.
Note that in order to use these options the server must be first
configured to expose location related info to the clients (i.e.
'show_multiple_locations' must be set to 'True").
I also added a mailmap entry for myself.
DocImpact
Closes-bug: #1271452
Co-Author: David Koo (koofoss) <david.koo@huawei.com>
Change-Id: Id1f320af05d9344645836359758e4aa227aafc69
It is Python2-specific, and should not be used in code intended to be portable.
Change-Id: Ibebef1a7e51a7444538275d22d444c926b8b4dec
Closes-Bug: 1284677
Some backend stores e.g. RBD, will fail if told to create an
image without a valid size (RBD will fail to write to a zero-size
image). Here we add support to allow the image size to be provided
when doing an upload. The result is that the upload content-length
will be set if available either from checking the supplied file
object or as provided by user.
Closes-Bug: 1220197
Change-Id: Ia1f2ea5680a139750d931591949b3e0058148b4b
Those modules have been combined in python 3. Therefore,
implementing the six module helps to have support to both versions.
Change-Id: I164a0f19790ff066d16d0cf4f0daa6f1097c848e
Closes-Bug: #1267181
six.iteritems() replaces dictionary.iteritems() on Python 2 and
dictionary.items() on Python 3.
six.string_types replaces basestring() in Python 2 and str
in Python 3.
Change-Id: Ia18510d167df35caec83626718010228e2140bc0
This patch will enable Glance client to query images by user
defined tags.
Implement bp image-query-by-tag
Implement bp glance-client-v2
Change-Id: I6f54630c5b7c9c567d85485ad4289284e5486814
image.patch returns a JSON schema Draft 10 (application/openstack-images-v2.1-json-patch)
while the glaceclient update method specify a Content-Type header
application/openstack-images-v2.0-json-patch with correspond to a JSON schemea Draft 4.
Fixes bug 1206095
Change-Id: I8c5a96f0e117a81b5b527a96ef45758fc69b518d
This adds support to the glance client library for removing
properties from an image. Properties to be removed must be
explicitly listed. There is no equivalent of the 'purge_props'
functionality in v1.
Also beefed up testing around create since some of the
functionality is the same as update.
Fixes bug 1206472
Change-Id: I16f4c39cdfc8a0cd07bede600461b7a289fbb080
A patch slipped in that modified the default image list limit in a
backwards-incompatible way. This change reverts that patch, but
preserves some of the formatting improvements.
Change-Id: I17ae5024896ca7b1064be66b9e47653e953771d6
Add a new module that contain generic wrapper for file and iterator, which
are used to wrap image to upload and the request body iterator in upload and
download cases repectively, to show and advance a pretty progress bar when this
laters are consumed, The progress bar is triggered by adding a --progress command
line argument to commands: image-create, image-download or image-update.
Change-Id: I2ba42fd0c58f4fa087adb568ec3f08246cae3759
bug fix: LP#1112309
blueprint: progressbar-when-uploading
Adds support for creating an image to the client library only, not the CLI.
Replaced reference to deprecated BaseException.message
Related to bp glance-client-v2
Change-Id: I8e3d09d89493368d22f7b1f69f79ebd2518e289d
This is a temporary solution. Increasing the default page size (which is being
used everytime, since the client is ignoring nova's --limit param). This should
decrease the number of queries that glance does when nova requests an image
detail list.
Bug 1200257
Change-Id: I7ed4521698570cbae9c20e69ddca8b11b57c65ad
Glanceclient implemented both functions before they landed into oslo.
Since both functions are already in oslo, it is now possible to pull
them in.
There's a small difference between glance's implementation and oslo's,
that is the later does not convert non-str objects - int, bool - to str
before trying to decode / encode them. This patch takes care of that
where necessary, more precisely, while encoding headers before doing a
new request.
Fixes bug: #1172253
Change-Id: I9a0dca31140bae28d8ec6aede515c5bb852b701b
Added the CLI option image-download to download an image via API V2.
Added utility function to save an image.
Added common iterator to validate the checksum.
Related to bp glance-client-v2
Change-Id: I0247f5a3462142dc5e9f3dc16cbe00c8e3d42f42
The v2 API no longer returns images in JSON containers like
'{"image": {...}}', so stop trying to decode the responses
as if it does.
Fix bug 1031185
Change-Id: I5209fe76445d4195b12944146a0ef190883f363f
* Use a recursive generator function to iterate over the image
container. The presence of next links are indicators to
continue pagination while their value drives the location of
the next page.
* A user can pass in --page-size on the command line, or page_size
when using the controller directly, to control how many images
are requested with each subsequent paginated request. Default page
size is 20.
* Add a flag (strict_url_check) for the FakeAPI class to control
whether it chops off query params when trying to match a request
to a fixture.
* Related to bp glance-client-v2.
Change-Id: Ib98e912a7af0bb570b4fd738733edd9b837d1a12
We will want this to be a generator as soon as we implement
pagination. Let's establish the interface now.
Related to bp glance-client-v2
Change-Id: Ib98e912a7af0bb570b4fd738733edd9b837d1a07
* Add warlock v0.1.0 as a dependency
* Generate a pythonic, self-validating Image model using warlock
* Add raw method to Schema model
* Related to bp glance-client-v2
Change-Id: Ib98e912a7af0bb570b4fd738733edd9b837d1a04
* Add image-create command
* Add tests for Image model, Controller.get, and Controller.list
* Related to bp glance-client-v2
Change-Id: Ib98e912a7af0bb570b4fd738733edd9b837d1a06
kairat_kushaev