37 Commits

Author SHA1 Message Date
Ian H Pittwood
c9d3ad98f7 Implement intermediary validation
Adds the options implemented in Spyglass for intermediary validation to
the plugin CLI

Related Change: https://review.opendev.org/#/c/678642/

Depends-On: Iaa385d265b027426f8e5f2376462ffb4c0d1d3fa
Change-Id: I15e348916c14927ff9a5e6a27e55a912f8dd3126
2019-08-29 08:47:37 -05:00
Ian H Pittwood
3b794d05ff Upgrade yapf to 0.28
Upgrades yapf to newest version, 0.28.0, and runs the formatter to
update all existing code.

Change-Id: I7b2cb0dfd80dbe7bad37f6046c7465f7abe71d1e
2019-08-15 13:12:35 -05:00
Ian H Pittwood
9713df8fec Loosen pins to match Spyglass
This change loosens pins on the XLS plugin dependencies to closer match
Spyglass. Loosening the pins allows Spyglass to have more choices in
package version when resolving dependencies.

Adds mandatory spyglass dependencies to test-requirementst since
Spyglass' requirements.txt has been removed. This will no longer be
necessary once PBR gets support for Pipfile.

Depends-On: I347ef6db18a44e1b88c811aa071ba8c60bccbd53
Change-Id: I8940b5afe1fce8d572ae90d66f6ae3cbefce9a70
2019-08-15 08:28:10 -05:00
Ian H. Pittwood
269ce7154f Standardize spyglass-plugin-xls code with YAPF
Implements documentation updates from [0] to standardize formatting for
all Airship projects.

[0] https://review.opendev.org/#/c/671291/

Change-Id: I723af0d91e24a306f05c06a3ddf456757489f7f0
2019-07-29 16:47:56 -05:00
Zuul
104c8b56a5 Merge "Move safety checks to new Zuul job" 2019-07-24 13:35:14 +00:00
Ian H Pittwood
2b585d1e40 Move safety checks to new Zuul job
Moves the dependency vulnerability checks from pep8 tox check to a new
Zuul job. This allows setting the job to not be voting as the database
used for safety has been flaky recently.

Changes basepython to python3 for safety and bandit jobs.

Related change: https://review.opendev.org/#/c/671847

Change-Id: I3bbc15aec795ff901e442e17d2ab974fd5da90ef
2019-07-23 08:42:42 -05:00
Ian H Pittwood
6eade386de Use new plugin base
Updates spyglass-plugin-xls to use the updated plugin base in Spyglass.

Related change: https://review.opendev.org/#/c/670898/

Depends-On: Ia3bc892994ff96cce5c1672fadf35ef2d1c4164b
Change-Id: If086d066610dfe2fd0384c7a3453940364ee1608
2019-07-22 17:38:48 +00:00
Ian H. Pittwood
8d3c35289e Use init to configure plugin
This change implements the removal of the configuration methods in
Spyglass. Class fields will now be set by the init method instead.

Depends-On: Ib26636f1eb4146902ee801af5bcce53d137be2ad
Change-Id: Ia7e53d900de14d1089b0323ddb08717b07b12de9
2019-07-18 14:23:28 -05:00
Zuul
d43d73da94 Merge "Centralize shared CLI options" 2019-07-17 15:27:47 +00:00
Zuul
07692b251d Merge "Revise formatting of excel specs" 2019-07-17 15:20:41 +00:00
Ian H. Pittwood
48b6cb98f5 Centralize shared CLI options
Moves shared CLI options to reference Spyglass main CLI definitions.

Depends-On: I9a6fe7a7d84fed71c372beea9cf7d74a2f6430b1
Change-Id: I735724fa1b5ac461c3cfd76fe96684a27999a69d
2019-07-17 15:03:08 +00:00
Ian H Pittwood
b9da6d41eb Revise formatting of excel specs
The previous excel spec seemed to lack a consistent structure. This
change redesigns the spec to more simply and generically find where data
is located in the spreadsheet.

Change-Id: I98c1553531897e8c623caa8a1b9334ea915a3f49
2019-07-16 09:07:28 -05:00
Ian H. Pittwood
3053b906a5 Upgrade Sphinx package
Sphinx 2.1.0 has a bug [0] that causes whitespace to be excluded in
outputted docs. This change updates Sphinx to peg any version >2.1.0.

Safety dependency vulnerability checks now will also cover the doc
requirements.txt.

[0] https://github.com/sphinx-doc/sphinx/issues/6440

Change-Id: I52b6eac208fd1c99194e2d30b6cf00431550a31d
2019-07-10 15:32:16 +00:00
Ian H. Pittwood
b0f7b952b4 Combines all exceptions into a single file
This change moves spyglass-plugin-xls's necessary exception definitions
locally into the spyglass-plugin-xls project and changes their base
to the SpyglassBaseException class. This change must be merged before
Spyglass's exceptions may be consolidated.

Related Change: https://review.opendev.org/#/c/667240/

Change-Id: I81c5ca2d9083aece3641bc8b5405dfd44baec810
2019-07-03 10:22:40 -05:00
Ian H Pittwood
e84b80c32d Testing for plugin
Adds tests for the Excel plugin's parser and extractor.

Enables pep8 and fmt checks on the tests directory.

Increases plugin test coverage to 94%, sets new minimum to 92%.

Fixes DNS and NTP server extraction with regex.

Updates file licenses.

Change-Id: I35ee97574e6d63b7a82cfa94caf79db5db9755e7
2019-07-01 10:13:47 -05:00
Ian H. Pittwood
d3212bd367 Use data objects for document generation
This change removes the intermediate step of converting the
SiteDocumentData object returned by the parser to a dictionary. This
change depends on [0], a related change that updates the Jinja2
templates and site processor to use data objects instead of a
dictionary.

[0] https://review.opendev.org/#/c/661092/

Depends-On: I66ebfeaf5d6ca76b6dee5a2285a74bad8b06b720
Change-Id: I991c9f6590752b36050dc5239d9ea3f3dfe8c5bc
2019-07-01 10:11:37 -05:00
Ian H. Pittwood
2ca6692909 Implements data objects in excel extractor
This change implements data objects from [0] into the spyglass XLS
plugin. With this change, the intermediary command will fully support
the data objects and documents will partially support them by converting
the objects to dictionaries for consumption in Jinja2.

[0] https://review.opendev.org/#/c/658917/

Depends-On: I101ad1ccbd95822965b8da8b6a644522eb2908e7
Change-Id: Ib95ef56cd8c5f4cddb9ff5c8c862ba7e323c8d49
2019-06-19 20:06:06 +00:00
Ian H. Pittwood
23892ce1ad Add unit tests for xls plugin CLI
This change implements unit tests for the Spyglass XLS plugin CLI.

Change-Id: I0761f8508f3134b1ffb77e78543a72bf1fcf5ca5
2019-06-17 11:06:31 -05:00
Ian H. Pittwood
93e78c472d Fixes issues with tox
A recent change updated the settings of yapf in Spyglass. This change
matches this recent change and fixes the directory used in tox. It also
fixes the directories used in tox.ini and removes unnecessary flake8
ignores.

Change-Id: I7036f885a3c45880b7b9be604bc6c1c533622134
2019-06-17 11:06:31 -05:00
Ian H. Pittwood
ec863a4f16 Fix circular Spyglass dependency
Adding PBR into spyglass-plugin-xls has caused a circular dependency.
Because Spyglass installs spyglass-plugin-xls it also now tries to
resolve spyglass-plugin-xls's requirements.txt which in turn requires
Spyglass to be installed. Since Spyglass is what's currently trying to
install, pip fails to resolve the plugin's dependencies. This change
moves the Spyglass requirement into test-requirements.txt. Spyglass will
always install this plugin so spyglass-plugin-xls should only need to
install Spyglass to run its own independent tests.

Change-Id: Ia5765d6b894cc1073c8be803d43f0b90cc448383
2019-06-17 14:48:31 +00:00
Zuul
c0ba5e77b8 Merge "Update plugin docs" 2019-06-14 20:28:28 +00:00
Zuul
dbeca0db91 Merge "Adds force option to manifest generation" 2019-06-14 20:24:20 +00:00
Ian H. Pittwood
934da3a3ea Adds force option to manifest generation
This change adds the "force" option to the XLS plugin, allowing users
to forgo manifest missing data validation.

Depends-On: Iff000eb173995156fbc6b44e621c59ba4dffae35
Change-Id: Icc9bf1bef0a6126e0d3915d3c809888c62086ee3
2019-06-12 14:23:47 -05:00
Ian H. Pittwood
d302ca306a Implement PBR for XLS plugin
Spyglass uses PBR for its packaging so it makes sense that its plugins
would do the same. This change makes the necessary updates to have
spyglass-plugin-xls use python build reasonableness.

Implements PBR in setup.py and setup.cfg

Moves YAPF configuration into setup.cfg

Change-Id: Ie0dab14bc266a0000566995c8fb2f9640fce7868
2019-06-11 16:14:32 -05:00
Zuul
5112a4d39e Merge "Pin and upgrade dependencies" 2019-06-10 15:58:23 +00:00
Ian H. Pittwood
8b11f843b0 Pin and upgrade dependencies
This change pins the test-requirements.txt dependencies and upgrades
two packages. PyYAML is upgraded to 5.1 to match Spyglass and openpyxl,
used to read Excel files, is upgraded to 2.6.2. Neither upgrade affects
the behavior of spyglass.

Change-Id: I0383ed56e62f64e2316925d1ba9d241b165ebdde
2019-06-10 10:44:54 -05:00
Zuul
789b929241 Merge "Adds Safety dependency vulnerability checks" 2019-06-06 20:55:23 +00:00
Zuul
7b3b429d54 Merge "Include examples in package" 2019-06-06 12:55:49 +00:00
Ian H. Pittwood
a28b11e087 Include examples in package
The examples folder should be kept inside of the package folder as it
won't be included if kept outside in the later versions of setuptools.

Change-Id: I14fb97e2f59571f0b20ae2c6aedfa62244bd6eb5
2019-06-05 08:52:07 -05:00
Ian H. Pittwood
db1fe14c0b Adds Safety dependency vulnerability checks
This change adds dependency vulnerability checks into tox. These checks
are performed by the Safety package which checks requirements files
against a vulnerability database for any issues. If any vulnerabilities
are found, tox will fail its pep8 env.

Change-Id: I6e356d1f55ced7df9490d34e06451acd9cbb6d70
2019-05-29 11:50:54 -05:00
Ian H. Pittwood
42b0bc5e47 Update plugin docs
Updates the XLS plugin docs using changes that were to be made by [0].

Changes old references to "tugboat" to "Spyglass XLS plugin".

[0]: https://review.opendev.org/#/c/650137/

Change-Id: I0df6a7f190548d0e0fa9a06d3b644dbc6cfb5280
2019-05-28 15:49:46 -05:00
Ian H. Pittwood
387de658dc Various fixes for plugin separation
There were a few missing changes from the repo that this project was
imported from. This change is meant to get the XLS plugin up to date
with any changes that were merged into Spyglass before the separation.

Relates to https://review.opendev.org/#/c/653555/

Change-Id: Ib2f75878b1a29e835cb8e2323aebe9d431c479e7
2019-05-16 14:13:40 -05:00
Ian H. Pittwood
471f47a711 Fix source directory name
The source directory for this package was originally named
"spyglass-plugin-xls", which caused it to not be able to be imported by
other projects. This change renames the directory to
"spyglass_plugin_xls" so it can be imported and utilized by Spyglass.

Change-Id: Ida8e7b8dc1d55bdf8cb38066ed72c40c0dcf474d
2019-05-14 11:26:46 -05:00
Andreas Jaeger
dff3326fc8 Add .gitreview and .zuul.yaml
Add the usual .gitreview file.

Add an initial zuul file so that jobs can merge, this contains only
noop-jobs, so no tests are run. A followup can add proper tests.

Change-Id: Idf5aa1c82419ddab718ca5267d5d250a2dfb64d9
2019-05-12 17:35:17 +02:00
Ian H. Pittwood
2a323424ee Added YAPF configuration
Added YAPF config recently implemented into Spyglass.

Reformatted code using YAPF.
2019-05-02 09:58:58 -05:00
Ian H. Pittwood
c74937c3d6 Removed zuul file 2019-04-29 10:57:19 -05:00
Ian H. Pittwood
6174c80f5d Initial commit
Adds code for tugboat plugin from Airship Spyglass.
2019-04-26 16:01:24 -05:00