jjb/jenkins-job-builder
Code Issues Proposed changes

registry: preload entrypoints before caching

Browse Source 
When profiling a JJB run against Wikimedia configuration files [0], I
noticed a lot of calls to package_resource trying to figure out the
version.

Years ago, I already added coded to cache the entrypoints, unfortunately
only the entry point definition got cached not the underlying function.
Thus each entry point ends up invoking a load() which dramatically slow
down JJB.

A low hanging fruit is to invoke load() and cache the returned function.
Doing so, the run time goes down from 28 seconds down to 17 seconds.

[0] https://gerrit.wikimedia.org/g/integration/config/+/master/jjb/
Change-Id: I336b556f28328d8acbdd71c43c8775d55c3720d8
This commit is contained in:
Antoine Musso 2019-10-28 17:41:44 +01:00
parent da1e227c8c
commit 9c78c09203
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
jenkins_jobs/registry.py
View File

@ -184,6 +184,7 @@ class ModuleRegistry(object):
# Look for a component function defined in an entry point
eps = self._entry_points_cache.get(component_list_type)
if eps is None:
logging.debug("Caching entrypoints for %s" % component_list_type)
module_eps = []
# auto build entry points by inferring from base component_types
mod = pkg_resources.EntryPoint(
@ -245,7 +246,7 @@ class ModuleRegistry(object):
"name: '{1}'".format(component_type, name)
)
eps[module_ep.name] = module_ep
eps[module_ep.name] = module_ep.load()
# cache both sets of entry points
self._entry_points_cache[component_list_type] = eps
@ -268,7 +269,7 @@ class ModuleRegistry(object):
# the arguments are interpolated into the real defn.
self.dispatch(component_type, xml_parent, b, component_data)
elif name in eps:
func = eps[name].load()
func = eps[name]
func(self, xml_parent, component_data)
else:
raise JenkinsJobsException(