Could you provide the following information: * Which OpenStack version you installed? (i.e. master, stable/stein, stable/rocky, etc.) * How did you install Zun (i.e. manual install, kolla, or devstack) * Your output of this command: pip freeze * You Zun config file (i.e. /etc/zun/zun.conf) python-neutronclient==6.7.0 python-novaclient==9.1.1 python-novnc===0.4-dfsg-1-20130425-git4973b9cc80 python-octaviaclient==1.4.0 python-openid==2.2.5 python-openstackclient==3.14.2 python-saharaclient==1.5.0 python-senlinclient==1.7.0 python-swiftclient==3.5.0 python-troveclient==2.14.0 python-zaqarclient==1.9.0 python-zunclient==1.1.0 pytimeparse==1.1.5 pytz==2018.3 pyxdg==0.25 PyYAML==3.12 rcssmin==1.0.6 redis==2.10.6 repoze.lru==0.7 requests==2.18.4 requestsexceptions==1.3.0 retrying==1.3.3 rfc3986==0.3.1 rjsmin==1.0.12 roman==2.0.0 Routes==2.4.1 ryu==4.15 sahara==8.0.0 sahara-dashboard==8.0.0 scgi==1.13 scrypt==0.8.0 SecretStorage==2.3.1 semantic-version==2.3.1 service-identity==16.0.0 setproctitle==1.1.10 simplegeneric==0.8.1 simplejson==3.13.2 singledispatch==3.4.0.3 six==1.11.0 SQLAlchemy==1.1.11 sqlalchemy-migrate==0.11.0 SQLAlchemy-Utils==0.32.21 sqlparse==0.2.4 statsd==3.2.1 stevedore==1.28.0 suds-jurko==0.7.dev0 swift==2.17.0 taskflow==3.1.0 Tempita==0.5.2 tenacity==4.8.0 testtools==2.3.0 tinyrpc==0.6 tooz==1.59.0 traceback2==1.4.0 trollius==2.0.1 trove==9.0.0 trove-dashboard==10.0.0 Twisted==17.9.0 typing==3.6.2 ujson==1.35 unicodecsv==0.14.1 unittest2==1.1.0 urllib3==1.22 vine==1.1.4 voluptuous==0.9.3 waitress==1.0.1 warlock==1.2.0 weakrefmethod==1.0.3 webencodings==0.5 WebOb==1.7.3 websocket-client==0.40.0 websockify==0.8.0 WebTest==2.0.28 Werkzeug==0.14.1 wrapt==1.9.0 WSME==0.9.2 xattr==0.9.2 xmltodict==0.11.0 yaql==1.1.3 zake==0.2.2 zope.interface==4.3.2 zun==1.0.2.dev13 zun-ui==1.0.1.dev30 # cat /etc/zun/zun.conf #THIS IS THE CONTROLLER [DEFAULT] # From oslo.log transport_url = rabbit://openstack:ce4e1a259c263b65d4d3@controller # If set to true, the logging level will be set to DEBUG instead of the default # INFO level. (boolean value) # Note: This option can be changed without restarting. #debug = false # The name of a logging configuration file. This file is appended to any # existing logging configuration files. For details about logging configuration # files, see the Python logging module documentation. Note that when logging # configuration files are used then all logging configuration is set in the # configuration file and other logging configuration options are ignored (for # example, logging_context_format_string). (string value) # Note: This option can be changed without restarting. # Deprecated group/name - [DEFAULT]/log_config #log_config_append = <None> # Defines the format string for %%(asctime)s in log records. Default: # %(default)s . This option is ignored if log_config_append is set. (string # value) #log_date_format = %Y-%m-%d %H:%M:%S # (Optional) Name of log file to send logging output to. If no default is set, # logging will go to stderr as defined by use_stderr. This option is ignored if # log_config_append is set. (string value) # Deprecated group/name - [DEFAULT]/logfile #log_file = <None> # (Optional) The base directory used for relative log_file paths. This option # is ignored if log_config_append is set. (string value) # Deprecated group/name - [DEFAULT]/logdir #log_dir = <None> # Uses logging handler designed to watch file system. When log file is moved or # removed this handler will open a new log file with specified path # instantaneously. It makes sense only if log_file option is specified and # Linux platform is used. This option is ignored if log_config_append is set. # (boolean value) #watch_log_file = false # Use syslog for logging. Existing syslog format is DEPRECATED and will be # changed later to honor RFC5424. This option is ignored if log_config_append # is set. (boolean value) #use_syslog = false # Enable journald for logging. If running in a systemd environment you may wish # to enable journal support. Doing so will use the journal native protocol # which includes structured metadata in addition to log messages.This option is # ignored if log_config_append is set. (boolean value) #use_journal = false # Syslog facility to receive log lines. This option is ignored if # log_config_append is set. (string value) #syslog_log_facility = LOG_USER # Use JSON formatting for logging. This option is ignored if log_config_append # is set. (boolean value) #use_json = false # Log output to standard error. This option is ignored if log_config_append is # set. (boolean value) #use_stderr = false # Format string to use for log messages with context. (string value) #logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s # Format string to use for log messages when context is undefined. (string # value) #logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s # Additional data to append to log message when logging level for the message # is DEBUG. (string value) #logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d # Prefix each line of exception output with this format. (string value) #logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s # Defines the format string for %(user_identity)s that is used in # logging_context_format_string. (string value) #logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s # List of package logging levels in logger=LEVEL pairs. This option is ignored # if log_config_append is set. (list value) #default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,oslo_messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN,keystoneauth=WARN,oslo.cache=INFO,dogpile.core.dogpile=INFO # Enables or disables publication of error events. (boolean value) #publish_errors = false # The format for an instance that is passed with the log message. (string # value) #instance_format = "[instance: %(uuid)s] " # The format for an instance UUID that is passed with the log message. (string # value) #instance_uuid_format = "[instance: %(uuid)s] " # Interval, number of seconds, of log rate limiting. (integer value) #rate_limit_interval = 0 # Maximum number of logged messages per rate_limit_interval. (integer value) #rate_limit_burst = 0 # Log level name used by rate limiting: CRITICAL, ERROR, INFO, WARNING, DEBUG # or empty string. Logs with level greater or equal to rate_limit_except_level # are not filtered. An empty string means that all levels are filtered. (string # value) #rate_limit_except_level = CRITICAL # Enables or disables fatal status of deprecations. (boolean value) #fatal_deprecations = false # From oslo.messaging # Size of RPC connection pool. (integer value) #rpc_conn_pool_size = 30 # The pool size limit for connections expiration policy (integer value) #conn_pool_min_size = 2 # The time-to-live in sec of idle connections in the pool (integer value) #conn_pool_ttl = 1200 # ZeroMQ bind address. Should be a wildcard (*), an ethernet interface, or IP. # The "host" option should point or resolve to this address. (string value) #rpc_zmq_bind_address = * # MatchMaker driver. (string value) # Possible values: # redis - <No description provided> # sentinel - <No description provided> # dummy - <No description provided> #rpc_zmq_matchmaker = redis # Number of ZeroMQ contexts, defaults to 1. (integer value) #rpc_zmq_contexts = 1 # Maximum number of ingress messages to locally buffer per topic. Default is # unlimited. (integer value) #rpc_zmq_topic_backlog = <None> # Directory for holding IPC sockets. (string value) #rpc_zmq_ipc_dir = /var/run/openstack # Name of this node. Must be a valid hostname, FQDN, or IP address. Must match # "host" option, if running Nova. (string value) #rpc_zmq_host = localhost # Number of seconds to wait before all pending messages will be sent after # closing a socket. The default value of -1 specifies an infinite linger # period. The value of 0 specifies no linger period. Pending messages shall be # discarded immediately when the socket is closed. Positive values specify an # upper bound for the linger period. (integer value) # Deprecated group/name - [DEFAULT]/rpc_cast_timeout #zmq_linger = -1 # The default number of seconds that poll should wait. Poll raises timeout # exception when timeout expired. (integer value) #rpc_poll_timeout = 1 # Expiration timeout in seconds of a name service record about existing target # ( < 0 means no timeout). (integer value) #zmq_target_expire = 300 # Update period in seconds of a name service record about existing target. # (integer value) #zmq_target_update = 180 # Use PUB/SUB pattern for fanout methods. PUB/SUB always uses proxy. (boolean # value) #use_pub_sub = false # Use ROUTER remote proxy. (boolean value) #use_router_proxy = false # This option makes direct connections dynamic or static. It makes sense only # with use_router_proxy=False which means to use direct connections for direct # message types (ignored otherwise). (boolean value) #use_dynamic_connections = false # How many additional connections to a host will be made for failover reasons. # This option is actual only in dynamic connections mode. (integer value) #zmq_failover_connections = 2 # Minimal port number for random ports range. (port value) # Minimum value: 0 # Maximum value: 65535 #rpc_zmq_min_port = 49153 # Maximal port number for random ports range. (integer value) # Minimum value: 1 # Maximum value: 65536 #rpc_zmq_max_port = 65536 # Number of retries to find free port number before fail with ZMQBindError. # (integer value) #rpc_zmq_bind_port_retries = 100 # Default serialization mechanism for serializing/deserializing # outgoing/incoming messages (string value) # Possible values: # json - <No description provided> # msgpack - <No description provided> #rpc_zmq_serialization = json # This option configures round-robin mode in zmq socket. True means not keeping # a queue when server side disconnects. False means to keep queue and messages # even if server is disconnected, when the server appears we send all # accumulated messages to it. (boolean value) #zmq_immediate = true # Enable/disable TCP keepalive (KA) mechanism. The default value of -1 (or any # other negative value) means to skip any overrides and leave it to OS default; # 0 and 1 (or any other positive value) mean to disable and enable the option # respectively. (integer value) #zmq_tcp_keepalive = -1 # The duration between two keepalive transmissions in idle condition. The unit # is platform dependent, for example, seconds in Linux, milliseconds in Windows # etc. The default value of -1 (or any other negative value and 0) means to # skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_idle = -1 # The number of retransmissions to be carried out before declaring that remote # end is not available. The default value of -1 (or any other negative value # and 0) means to skip any overrides and leave it to OS default. (integer # value) #zmq_tcp_keepalive_cnt = -1 # The duration between two successive keepalive retransmissions, if # acknowledgement to the previous keepalive transmission is not received. The # unit is platform dependent, for example, seconds in Linux, milliseconds in # Windows etc. The default value of -1 (or any other negative value and 0) # means to skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_intvl = -1 # Maximum number of (green) threads to work concurrently. (integer value) #rpc_thread_pool_size = 100 # Expiration timeout in seconds of a sent/received message after which it is # not tracked anymore by a client/server. (integer value) #rpc_message_ttl = 300 # Wait for message acknowledgements from receivers. This mechanism works only # via proxy without PUB/SUB. (boolean value) #rpc_use_acks = false # Number of seconds to wait for an ack from a cast/call. After each retry # attempt this timeout is multiplied by some specified multiplier. (integer # value) #rpc_ack_timeout_base = 15 # Number to multiply base ack timeout by after each retry attempt. (integer # value) #rpc_ack_timeout_multiplier = 2 # Default number of message sending attempts in case of any problems occurred: # positive value N means at most N retries, 0 means no retries, None or -1 (or # any other negative values) mean to retry forever. This option is used only if # acknowledgments are enabled. (integer value) #rpc_retry_attempts = 3 # List of publisher hosts SubConsumer can subscribe on. This option has higher # priority then the default publishers list taken from the matchmaker. (list # value) #subscribe_on = # Size of executor thread pool when executor is threading or eventlet. (integer # value) # Deprecated group/name - [DEFAULT]/rpc_thread_pool_size #executor_thread_pool_size = 64 # Seconds to wait for a response from a call. (integer value) #rpc_response_timeout = 60 # The network address and optional user credentials for connecting to the # messaging backend, in URL format. The expected format is: # driver://[user:pass@]host:port[,[userN:passN@]hostN:portN]/virtual_host?query # Example: rabbit://rabbitmq:[email protected]:5672// # For full details on the fields in the URL see the documentation of # oslo_messaging.TransportURL at # https://docs.openstack.org/oslo.messaging/latest/reference/transport.html # (string value) #transport_url = <None> # DEPRECATED: The messaging driver to use, defaults to rabbit. Other drivers # include amqp and zmq. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rpc_backend = rabbit # The default exchange under which topics are scoped. May be overridden by an # exchange name specified in the transport_url option. (string value) #control_exchange = openstack # From oslo.service.periodic_task # Some periodic tasks can be run in a separate process. Should we run them # here? (boolean value) #run_external_periodic_tasks = true # From oslo.service.service # Enable eventlet backdoor. Acceptable values are 0, <port>, and # <start>:<end>, where 0 results in listening on a random tcp port number; # <port> results in listening on the specified port number (and not enabling # backdoor if that port is in use); and <start>:<end> results in listening on # the smallest unused port number within the specified range of port numbers. # The chosen port is displayed in the service's log file. (string value) #backdoor_port = <None> # Enable eventlet backdoor, using the provided path as a unix socket that can # receive connections. This option is mutually exclusive with 'backdoor_port' # in that only one should be provided. If both are provided then the existence # of this option overrides the usage of that option. (string value) #backdoor_socket = <None> # Enables or disables logging values of all registered options when starting a # service (at DEBUG level). (boolean value) #log_options = true # Specify a timeout after which a gracefully shutdown server will exit. Zero # value means endless wait. (integer value) #graceful_shutdown_timeout = 60 # From zun.conf # Defines which driver to use for controlling container. # Possible values: # * ``docker.driver.DockerDriver`` # Services which consume this: # * ``zun-compute`` # Interdependencies to other options: # * None # (string value) #container_driver = docker.driver.DockerDriver # Time to sleep (in seconds) during waiting for an event. (integer value) #default_sleep_time = 1 # Maximum time (in seconds) to wait for an event. (integer value) #default_timeout = 600 # Define the cpusets to be excluded from pinning (string value) #floating_cpu_set = # Whether to use infra container. If set to True, # Zun will create an infra container that serves as a placeholder of a few # Linux namespaces (i.e. network namespace). Then, one or multiple containers # could join the namespaces of the infra container thus sharing resources # inside # the sandbox (i.e. the network interface). This is typically used to group # a set of high-coupled containers into a unit. If set to False, infra # container # won't be created. # (boolean value) #use_sandbox = false # Define the runtime to create container with. Default value # in Zun is ``runc``. (string value) #container_runtime = runc # The default memory swap size in MB (default is -1 which enable unlimited # swap). (integer value) #default_memory_swap = -1 # MySQL engine to use. (string value) #mysql_engine = InnoDB # Defines the list of image driver to use for downloading image. # Possible values: # * ``docker`` # * ``glance`` # Services which consume this: # * ``zun-compute`` # Interdependencies to other options: # * None # (list value) #image_driver_list = glance,docker # Container image for sandbox container. (string value) #sandbox_image = kubernetes/pause # Image driver for sandbox container. (string value) #sandbox_image_driver = docker # Image pull policy for sandbox image. (string value) #sandbox_image_pull_policy = ifnotpresent # The IP address which the host is using to connect to the management network. # Possible values: # * String with valid IP address. Default is IPv4 address of this host. # Related options: # * docker_remote_api_host # * etcd_host # * wsproxy_host # * host_ip # * my_block_storage_ip # (string value) #my_ip = <host_ipv4> # Hostname, FQDN or IP address of this host. This can be an opaque identifier. # It is not necessarily a hostname, FQDN, or IP address. However, the node name # must be valid within an AMQP key, and if using ZeroMQ, a valid hostname, # FQDN, or IP address. # Possible values: # * String with hostname, FQDN or IP address. Default is hostname of this host. # (string value) #host = <current_hostname> # The IP address which is used to connect to the block storage network. # Possible values: # * String with valid IP address. Default is IP address of this host. # Related options: # * my_ip - if my_block_storage_ip is not set, then my_ip value is used. # (string value) #my_block_storage_ip = $my_ip # Directory where the zun python module is installed. (string value) #pybasedir = /usr/local/lib/python2.7/dist-packages/zun # Directory where zun binaries are installed. (string value) #bindir = $pybasedir/bin # Top-level directory for maintaining zun's state. (string value) #state_path = $pybasedir # Max interval size between periodic tasks execution in seconds. (integer # value) #periodic_interval_max = 60 # Max interval size between periodic tasks execution in seconds. (integer # value) #service_down_time = 180 # Interval to sync container states between the database and the docker. # The interval that Zun checks the actual container state and # the state that Zun has recorded in its database. If they are inconsistent, # Zun will update the database according to the actual container state. # Possible values: # * 0: Will run at the default periodic interval. # * Any value < 0: Disables the option. # * Any positive integer in seconds. # (integer value) #sync_container_state_interval = 60 # Path to the rootwrap configuration file to use for running commands as root. # (string value) #rootwrap_config = /etc/zun/rootwrap.conf [api] # From zun.conf host_ip = 192.168.0.1 port = 9517 # The port for the zun API server. (port value) # Minimum value: 0 # Maximum value: 65535 #port = 9517 # The listen IP for the zun API server. The default is ``$my_ip``, the IP # address of this host. (IP address value) #host_ip = $my_ip # Enable the integrated stand-alone API to service requests via HTTPS instead # of HTTP. If there is a front-end service performing HTTPS offloading from the # service, this option should be False; note, you will want to change public # API endpoint to represent SSL termination URL with 'public_endpoint' option. # (boolean value) #enable_ssl_api = false # Number of workers for zun-api service. The default will be the number of CPUs # available. (integer value) #workers = <None> # The maximum number of items returned in a single response from a collection # resource. (integer value) #max_limit = 1000 # Configuration file for WSGI definition of API. (string value) #api_paste_config = api-paste.ini # Enable image validation. (boolean value) #enable_image_validation = true [cinder_client] # From zun.conf # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Version of Cinder API to use in cinderclient. (string value) #api_version = 3 # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [compute] # From zun.conf # The queue to add compute tasks to. (string value) #topic = zun-compute # Sets the scope of the check for unique container names. # The default doesn't check for unique names. If a scope for the name check is # set, a launch of a new container with a duplicate name will result in an # ''ContainerAlreadyExists'' error. The uniqueness is case-insensitive. # Setting this option can increase the usability for end users as they don't # have to distinguish among containers with the same name by their IDs. # Possible values: # * '': An empty value means that no uniqueness check is done and duplicate # names are possible. # * "project": The container name check is done only for containers within the # same project. # * "global": The container name check is done for all containers regardless of # the project. # (string value) # Possible values: # '' - <No description provided> # project - <No description provided> # global - <No description provided> #unique_container_name_scope = # restart the containers which are runningbefore the host reboots. (boolean # value) #resume_container_state = true [cors] # From oslo.middleware.cors # Indicate whether this resource may be shared with the domain received in the # requests "origin" header. Format: "<protocol>://<host>[:<port>]", no trailing # slash. Example: https://horizon.example.com (list value) #allowed_origin = <None> # Indicate that the actual request can include user credentials (boolean value) #allow_credentials = true # Indicate which headers are safe to expose to the API. Defaults to HTTP Simple # Headers. (list value) #expose_headers = # Maximum cache age of CORS preflight requests. (integer value) #max_age = 3600 # Indicate which methods can be used during the actual request. (list value) #allow_methods = OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,PATCH # Indicate which header field names may be used during the actual request. # (list value) #allow_headers = [database] # From oslo.db connection = mysql+pymysql://zun:zun@controller/zun # If True, SQLite uses synchronous mode. (boolean value) #sqlite_synchronous = true # The back end to use for the database. (string value) # Deprecated group/name - [DEFAULT]/db_backend #backend = sqlalchemy # The SQLAlchemy connection string to use to connect to the database. (string # value) # Deprecated group/name - [DEFAULT]/sql_connection # Deprecated group/name - [DATABASE]/sql_connection # Deprecated group/name - [sql]/connection #connection = <None> # The SQLAlchemy connection string to use to connect to the slave database. # (string value) #slave_connection = <None> # The SQL mode to be used for MySQL sessions. This option, including the # default, overrides any server-set SQL mode. To use whatever SQL mode is set # by the server configuration, set this to no value. Example: mysql_sql_mode= # (string value) #mysql_sql_mode = TRADITIONAL # If True, transparently enables support for handling MySQL Cluster (NDB). # (boolean value) #mysql_enable_ndb = false # Connections which have been present in the connection pool longer than this # number of seconds will be replaced with a new one the next time they are # checked out from the pool. (integer value) # Deprecated group/name - [DATABASE]/idle_timeout # Deprecated group/name - [database]/idle_timeout # Deprecated group/name - [DEFAULT]/sql_idle_timeout # Deprecated group/name - [DATABASE]/sql_idle_timeout # Deprecated group/name - [sql]/idle_timeout #connection_recycle_time = 3600 # Minimum number of SQL connections to keep open in a pool. (integer value) # Deprecated group/name - [DEFAULT]/sql_min_pool_size # Deprecated group/name - [DATABASE]/sql_min_pool_size #min_pool_size = 1 # Maximum number of SQL connections to keep open in a pool. Setting a value of # 0 indicates no limit. (integer value) # Deprecated group/name - [DEFAULT]/sql_max_pool_size # Deprecated group/name - [DATABASE]/sql_max_pool_size #max_pool_size = 5 # Maximum number of database connection retries during startup. Set to -1 to # specify an infinite retry count. (integer value) # Deprecated group/name - [DEFAULT]/sql_max_retries # Deprecated group/name - [DATABASE]/sql_max_retries #max_retries = 10 # Interval between retries of opening a SQL connection. (integer value) # Deprecated group/name - [DEFAULT]/sql_retry_interval # Deprecated group/name - [DATABASE]/reconnect_interval #retry_interval = 10 # If set, use this value for max_overflow with SQLAlchemy. (integer value) # Deprecated group/name - [DEFAULT]/sql_max_overflow # Deprecated group/name - [DATABASE]/sqlalchemy_max_overflow #max_overflow = 50 # Verbosity of SQL debugging information: 0=None, 100=Everything. (integer # value) # Minimum value: 0 # Maximum value: 100 # Deprecated group/name - [DEFAULT]/sql_connection_debug #connection_debug = 0 # Add Python stack traces to SQL as comment strings. (boolean value) # Deprecated group/name - [DEFAULT]/sql_connection_trace #connection_trace = false # If set, use this value for pool_timeout with SQLAlchemy. (integer value) # Deprecated group/name - [DATABASE]/sqlalchemy_pool_timeout #pool_timeout = <None> # Enable the experimental use of database reconnect on connection lost. # (boolean value) #use_db_reconnect = false # Seconds between retries of a database transaction. (integer value) #db_retry_interval = 1 # If True, increases the interval between retries of a database operation up to # db_max_retry_interval. (boolean value) #db_inc_retry_interval = true # If db_inc_retry_interval is set, the maximum seconds between retries of a # database operation. (integer value) #db_max_retry_interval = 10 # Maximum retries in case of connection error or deadlock error before error is # raised. Set to -1 to specify an infinite retry count. (integer value) #db_max_retries = 20 [docker] # From zun.conf # Docker remote api version. Override it according to specific docker api # version in your environment. (string value) #docker_remote_api_version = 1.26 # Default timeout in seconds for docker client operations. (integer value) #default_timeout = 60 # API endpoint of docker daemon (string value) #api_url = unix:///var/run/docker.sock # Remote API endpoint of docker daemon (string value) #docker_remote_api_url = tcp://$docker_remote_api_host:$docker_remote_api_port # If set, ignore any SSL validation issues (boolean value) #api_insecure = false # Location of CA certificates file for securing docker api requests # (tlscacert). (string value) #ca_file = <None> # Location of TLS certificate file for securing docker api requests (tlscert). # (string value) #cert_file = <None> # Location of TLS private key file for securing docker api requests (tlskey). # (string value) #key_file = <None> # Defines the remote api host for the docker daemon. (string value) #docker_remote_api_host = $my_ip # Defines the remote api port for the docker daemon. (string value) #docker_remote_api_port = 2375 # Timeout in seconds for executing a command in a docker container. (integer # value) #execute_timeout = 5 [etcd] # From zun.conf # Host IP address on which etcd service running. The default is ``$my_ip``, the # IP address of this host. (unknown value) #etcd_host = $my_ip # Port on which etcd listen client request. (port value) # Minimum value: 0 # Maximum value: 65535 #etcd_port = 2379 [glance] # From zun.conf # Shared directory where glance images located. If specified, docker will try # to load the image from the shared directory by image ID. (string value) #images_directory = $state_path/images [glance_client] # From zun.conf # Region in Identity service catalog to use for communication with the # OpenStack service. (string value) #region_name = <None> # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Version of Glance API to use in glanceclient. (string value) #api_version = 2 # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [keystone_auth] # From zun.conf memcached_servers = controller:11211 www_authenticate_uri = http://controller:5000 project_domain_name = default project_name = service user_domain_name = default password = zun username = zun auth_url = http://controller:5000 auth_type = password auth_version = v3 auth_protocol = http service_token_roles_required = True endpoint_type = internalURL # Authentication type to load (string value) # Deprecated group/name - [keystone_auth]/auth_plugin #auth_type = <None> # Config Section from which to load plugin specific options (string value) #auth_section = <None> # Authentication URL (string value) #auth_url = <None> # Scope for system operations (string value) #system_scope = <None> # Domain ID to scope to (string value) #domain_id = <None> # Domain name to scope to (string value) #domain_name = <None> # Project ID to scope to (string value) # Deprecated group/name - [keystone_auth]/tenant_id #project_id = <None> # Project name to scope to (string value) # Deprecated group/name - [keystone_auth]/tenant_name #project_name = <None> # Domain ID containing project (string value) #project_domain_id = <None> # Domain name containing project (string value) #project_domain_name = <None> # Trust ID (string value) #trust_id = <None> # Optional domain ID to use with v3 and v2 parameters. It will be used for both # the user and project domain in v3 and ignored in v2 authentication. (string # value) #default_domain_id = <None> # Optional domain name to use with v3 API and v2 parameters. It will be used # for both the user and project domain in v3 and ignored in v2 authentication. # (string value) #default_domain_name = <None> # User id (string value) #user_id = <None> # Username (string value) # Deprecated group/name - [keystone_auth]/user_name #username = <None> # User's domain id (string value) #user_domain_id = <None> # User's domain name (string value) #user_domain_name = <None> # User's password (string value) #password = <None> [keystone_authtoken] # From keystonemiddleware.auth_token memcached_servers = controller:11211 www_authenticate_uri = http://controller:5000 project_domain_name = default project_name = service user_domain_name = default password = zun username = zun auth_url = http://controller:5000 auth_type = password auth_version = v3 auth_protocol = http service_token_roles_required = True endpoint_type = internalURL # Complete "public" Identity API endpoint. This endpoint should not be an # "admin" endpoint, as it should be accessible by all end users. # Unauthenticated clients are redirected to this endpoint to authenticate. # Although this endpoint should ideally be unversioned, client support in the # wild varies. If you're using a versioned v2 endpoint here, then this should # *not* be the same endpoint the service user utilizes for validating tokens, # because normal end users may not be able to reach that endpoint. (string # value) # Deprecated group/name - [keystone_authtoken]/auth_uri #www_authenticate_uri = <None> # DEPRECATED: Complete "public" Identity API endpoint. This endpoint should not # be an "admin" endpoint, as it should be accessible by all end users. # Unauthenticated clients are redirected to this endpoint to authenticate. # Although this endpoint should ideally be unversioned, client support in the # wild varies. If you're using a versioned v2 endpoint here, then this should # *not* be the same endpoint the service user utilizes for validating tokens, # because normal end users may not be able to reach that endpoint. This option # is deprecated in favor of www_authenticate_uri and will be removed in the S # release. (string value) # This option is deprecated for removal since Queens. # Its value may be silently ignored in the future. # Reason: The auth_uri option is deprecated in favor of www_authenticate_uri # and will be removed in the S release. #auth_uri = <None> # API version of the admin Identity API endpoint. (string value) #auth_version = <None> # Do not handle authorization requests within the middleware, but delegate the # authorization decision to downstream WSGI components. (boolean value) #delay_auth_decision = false # Request timeout value for communicating with Identity API server. (integer # value) #http_connect_timeout = <None> # How many times are we trying to reconnect when communicating with Identity # API Server. (integer value) #http_request_max_retries = 3 # Request environment key where the Swift cache object is stored. When # auth_token middleware is deployed with a Swift cache, use this option to have # the middleware share a caching backend with swift. Otherwise, use the # ``memcached_servers`` option instead. (string value) #cache = <None> # Required if identity server requires client certificate (string value) #certfile = <None> # Required if identity server requires client certificate (string value) #keyfile = <None> # A PEM encoded Certificate Authority to use when verifying HTTPs connections. # Defaults to system CAs. (string value) #cafile = <None> # Verify HTTPS connections. (boolean value) #insecure = false # The region in which the identity server can be found. (string value) #region_name = <None> # DEPRECATED: Directory used to cache files related to PKI tokens. This option # has been deprecated in the Ocata release and will be removed in the P # release. (string value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #signing_dir = <None> # Optionally specify a list of memcached server(s) to use for caching. If left # undefined, tokens will instead be cached in-process. (list value) # Deprecated group/name - [keystone_authtoken]/memcache_servers #memcached_servers = <None> # In order to prevent excessive effort spent validating tokens, the middleware # caches previously-seen tokens for a configurable duration (in seconds). Set # to -1 to disable caching completely. (integer value) #token_cache_time = 300 # DEPRECATED: Determines the frequency at which the list of revoked tokens is # retrieved from the Identity service (in seconds). A high number of revocation # events combined with a low cache duration may significantly reduce # performance. Only valid for PKI tokens. This option has been deprecated in # the Ocata release and will be removed in the P release. (integer value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #revocation_cache_time = 10 # (Optional) If defined, indicate whether token data should be authenticated or # authenticated and encrypted. If MAC, token data is authenticated (with HMAC) # in the cache. If ENCRYPT, token data is encrypted and authenticated in the # cache. If the value is not one of these options or empty, auth_token will # raise an exception on initialization. (string value) # Possible values: # None - <No description provided> # MAC - <No description provided> # ENCRYPT - <No description provided> #memcache_security_strategy = None # (Optional, mandatory if memcache_security_strategy is defined) This string is # used for key derivation. (string value) #memcache_secret_key = <None> # (Optional) Number of seconds memcached server is considered dead before it is # tried again. (integer value) #memcache_pool_dead_retry = 300 # (Optional) Maximum total number of open connections to every memcached # server. (integer value) #memcache_pool_maxsize = 10 # (Optional) Socket timeout in seconds for communicating with a memcached # server. (integer value) #memcache_pool_socket_timeout = 3 # (Optional) Number of seconds a connection to memcached is held unused in the # pool before it is closed. (integer value) #memcache_pool_unused_timeout = 60 # (Optional) Number of seconds that an operation will wait to get a memcached # client connection from the pool. (integer value) #memcache_pool_conn_get_timeout = 10 # (Optional) Use the advanced (eventlet safe) memcached client pool. The # advanced pool will only work under python 2.x. (boolean value) #memcache_use_advanced_pool = false # (Optional) Indicate whether to set the X-Service-Catalog header. If False, # middleware will not ask for service catalog on token validation and will not # set the X-Service-Catalog header. (boolean value) #include_service_catalog = true # Used to control the use and type of token binding. Can be set to: "disabled" # to not check token binding. "permissive" (default) to validate binding # information if the bind type is of a form known to the server and ignore it # if not. "strict" like "permissive" but if the bind type is unknown the token # will be rejected. "required" any form of token binding is needed to be # allowed. Finally the name of a binding method that must be present in tokens. # (string value) #enforce_token_bind = permissive # DEPRECATED: If true, the revocation list will be checked for cached tokens. # This requires that PKI tokens are configured on the identity server. (boolean # value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #check_revocations_for_cached = false # DEPRECATED: Hash algorithms to use for hashing PKI tokens. This may be a # single algorithm or multiple. The algorithms are those supported by Python # standard hashlib.new(). The hashes will be tried in the order given, so put # the preferred one first for performance. The result of the first hash will be # stored in the cache. This will typically be set to multiple values only while # migrating from a less secure algorithm to a more secure one. Once all the old # tokens are expired this option should be set to a single value for better # performance. (list value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #hash_algorithms = md5 # A choice of roles that must be present in a service token. Service tokens are # allowed to request that an expired token can be used and so this check should # tightly control that only actual services should be sending this token. Roles # here are applied as an ANY check so any role in this list must be present. # For backwards compatibility reasons this currently only affects the # allow_expired check. (list value) #service_token_roles = service # For backwards compatibility reasons we must let valid service tokens pass # that don't pass the service_token_roles check as valid. Setting this true # will become the default in a future release and should be enabled if # possible. (boolean value) #service_token_roles_required = false # Prefix to prepend at the beginning of the path. Deprecated, use identity_uri. # (string value) #auth_admin_prefix = # Host providing the admin Identity API endpoint. Deprecated, use identity_uri. # (string value) #auth_host = 127.0.0.1 # Port of the admin Identity API endpoint. Deprecated, use identity_uri. # (integer value) #auth_port = 35357 # Protocol of the admin Identity API endpoint. Deprecated, use identity_uri. # (string value) # Possible values: # http - <No description provided> # https - <No description provided> #auth_protocol = https # Complete admin Identity API endpoint. This should specify the unversioned # root endpoint e.g. https://localhost:35357/ (string value) #identity_uri = <None> # This option is deprecated and may be removed in a future release. Single # shared secret with the Keystone configuration used for bootstrapping a # Keystone installation, or otherwise bypassing the normal authentication # process. This option should not be used, use `admin_user` and # `admin_password` instead. (string value) #admin_token = <None> # Service username. (string value) #admin_user = <None> # Service user password. (string value) #admin_password = <None> # Service tenant name. (string value) #admin_tenant_name = admin # Authentication type to load (string value) # Deprecated group/name - [keystone_authtoken]/auth_plugin #auth_type = <None> # Config Section from which to load plugin specific options (string value) #auth_section = <None> [matchmaker_redis] # From oslo.messaging # DEPRECATED: Host to locate redis. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #host = 127.0.0.1 # DEPRECATED: Use this port to connect to redis host. (port value) # Minimum value: 0 # Maximum value: 65535 # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #port = 6379 # DEPRECATED: Password for Redis server (optional). (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #password = # DEPRECATED: List of Redis Sentinel hosts (fault tolerance mode), e.g., # [host:port, host1:port ... ] (list value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #sentinel_hosts = # Redis replica set name. (string value) #sentinel_group_name = oslo-messaging-zeromq # Time in ms to wait between connection attempts. (integer value) #wait_timeout = 2000 # Time in ms to wait before the transaction is killed. (integer value) #check_timeout = 20000 # Timeout in ms on blocking socket operations. (integer value) #socket_timeout = 10000 [network] # From zun.conf # Defines which driver to use for container network. (string value) #driver = kuryr # The network plugin driver name, you can find it by docker plugin list. # (string value) #driver_name = kuryr [neutron_client] # From zun.conf # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [nova_client] # From zun.conf # Version of Nova API to use in novaclient. (string value) #api_version = 2.37 # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [oslo_concurrency] # From oslo.concurrency lock_path = /var/lib/zun/tmp # Enables or disables inter-process locks. (boolean value) #disable_process_locking = false # Directory to use for lock files. For security, the specified directory # should only be writable by the user running the processes that need locking. # Defaults to environment variable OSLO_LOCK_PATH. If OSLO_LOCK_PATH is not set # in the environment, use the Python tempfile.gettempdir function to find a # suitable location. If external locks are used, a lock path must be set. # (string value) #lock_path = /tmp [oslo_messaging_amqp] # From oslo.messaging # Name for the AMQP container. must be globally unique. Defaults to a generated # UUID (string value) #container_name = <None> # Timeout for inactive connections (in seconds) (integer value) #idle_timeout = 0 # Debug: dump AMQP frames to stdout (boolean value) #trace = false # Attempt to connect via SSL. If no other ssl-related parameters are given, it # will use the system's CA-bundle to verify the server's certificate. (boolean # value) #ssl = false # CA certificate PEM file used to verify the server's certificate (string # value) #ssl_ca_file = # Self-identifying certificate PEM file for client authentication (string # value) #ssl_cert_file = # Private key PEM file used to sign ssl_cert_file certificate (optional) # (string value) #ssl_key_file = # Password for decrypting ssl_key_file (if encrypted) (string value) #ssl_key_password = <None> # By default SSL checks that the name in the server's certificate matches the # hostname in the transport_url. In some configurations it may be preferable to # use the virtual hostname instead, for example if the server uses the Server # Name Indication TLS extension (rfc6066) to provide a certificate per virtual # host. Set ssl_verify_vhost to True if the server's SSL certificate uses the # virtual host name instead of the DNS name. (boolean value) #ssl_verify_vhost = false # DEPRECATED: Accept clients using either SSL or plain TCP (boolean value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Not applicable - not a SSL server #allow_insecure_clients = false # Space separated list of acceptable SASL mechanisms (string value) #sasl_mechanisms = # Path to directory that contains the SASL configuration (string value) #sasl_config_dir = # Name of configuration file (without .conf suffix) (string value) #sasl_config_name = # SASL realm to use if no realm present in username (string value) #sasl_default_realm = # DEPRECATED: User name for message broker authentication (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Should use configuration option transport_url to provide the # username. #username = # DEPRECATED: Password for message broker authentication (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Should use configuration option transport_url to provide the # password. #password = # Seconds to pause before attempting to re-connect. (integer value) # Minimum value: 1 #connection_retry_interval = 1 # Increase the connection_retry_interval by this many seconds after each # unsuccessful failover attempt. (integer value) # Minimum value: 0 #connection_retry_backoff = 2 # Maximum limit for connection_retry_interval + connection_retry_backoff # (integer value) # Minimum value: 1 #connection_retry_interval_max = 30 # Time to pause between re-connecting an AMQP 1.0 link that failed due to a # recoverable error. (integer value) # Minimum value: 1 #link_retry_delay = 10 # The maximum number of attempts to re-send a reply message which failed due to # a recoverable error. (integer value) # Minimum value: -1 #default_reply_retry = 0 # The deadline for an rpc reply message delivery. (integer value) # Minimum value: 5 #default_reply_timeout = 30 # The deadline for an rpc cast or call message delivery. Only used when caller # does not provide a timeout expiry. (integer value) # Minimum value: 5 #default_send_timeout = 30 # The deadline for a sent notification message delivery. Only used when caller # does not provide a timeout expiry. (integer value) # Minimum value: 5 #default_notify_timeout = 30 # The duration to schedule a purge of idle sender links. Detach link after # expiry. (integer value) # Minimum value: 1 #default_sender_link_timeout = 600 # Indicates the addressing mode used by the driver. # Permitted values: # 'legacy' - use legacy non-routable addressing # 'routable' - use routable addresses # 'dynamic' - use legacy addresses if the message bus does not support routing # otherwise use routable addressing (string value) #addressing_mode = dynamic # Enable virtual host support for those message buses that do not natively # support virtual hosting (such as qpidd). When set to true the virtual host # name will be added to all message bus addresses, effectively creating a # private 'subnet' per virtual host. Set to False if the message bus supports # virtual hosting using the 'hostname' field in the AMQP 1.0 Open performative # as the name of the virtual host. (boolean value) #pseudo_vhost = true # address prefix used when sending to a specific server (string value) #server_request_prefix = exclusive # address prefix used when broadcasting to all servers (string value) #broadcast_prefix = broadcast # address prefix when sending to any server in group (string value) #group_request_prefix = unicast # Address prefix for all generated RPC addresses (string value) #rpc_address_prefix = openstack.org/om/rpc # Address prefix for all generated Notification addresses (string value) #notify_address_prefix = openstack.org/om/notify # Appended to the address prefix when sending a fanout message. Used by the # message bus to identify fanout messages. (string value) #multicast_address = multicast # Appended to the address prefix when sending to a particular RPC/Notification # server. Used by the message bus to identify messages sent to a single # destination. (string value) #unicast_address = unicast # Appended to the address prefix when sending to a group of consumers. Used by # the message bus to identify messages that should be delivered in a round- # robin fashion across consumers. (string value) #anycast_address = anycast # Exchange name used in notification addresses. # Exchange name resolution precedence: # Target.exchange if set # else default_notification_exchange if set # else control_exchange if set # else 'notify' (string value) #default_notification_exchange = <None> # Exchange name used in RPC addresses. # Exchange name resolution precedence: # Target.exchange if set # else default_rpc_exchange if set # else control_exchange if set # else 'rpc' (string value) #default_rpc_exchange = <None> # Window size for incoming RPC Reply messages. (integer value) # Minimum value: 1 #reply_link_credit = 200 # Window size for incoming RPC Request messages (integer value) # Minimum value: 1 #rpc_server_credit = 100 # Window size for incoming Notification messages (integer value) # Minimum value: 1 #notify_server_credit = 100 # Send messages of this type pre-settled. # Pre-settled messages will not receive acknowledgement # from the peer. Note well: pre-settled messages may be # silently discarded if the delivery fails. # Permitted values: # 'rpc-call' - send RPC Calls pre-settled # 'rpc-reply'- send RPC Replies pre-settled # 'rpc-cast' - Send RPC Casts pre-settled # 'notify' - Send Notifications pre-settled # (multi valued) #pre_settled = rpc-cast #pre_settled = rpc-reply [oslo_messaging_kafka] # From oslo.messaging # DEPRECATED: Default Kafka broker Host (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #kafka_default_host = localhost # DEPRECATED: Default Kafka broker Port (port value) # Minimum value: 0 # Maximum value: 65535 # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #kafka_default_port = 9092 # Max fetch bytes of Kafka consumer (integer value) #kafka_max_fetch_bytes = 1048576 # Default timeout(s) for Kafka consumers (floating point value) #kafka_consumer_timeout = 1.0 # DEPRECATED: Pool Size for Kafka Consumers (integer value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Driver no longer uses connection pool. #pool_size = 10 # DEPRECATED: The pool size limit for connections expiration policy (integer # value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Driver no longer uses connection pool. #conn_pool_min_size = 2 # DEPRECATED: The time-to-live in sec of idle connections in the pool (integer # value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Driver no longer uses connection pool. #conn_pool_ttl = 1200 # Group id for Kafka consumer. Consumers in one group will coordinate message # consumption (string value) #consumer_group = oslo_messaging_consumer # Upper bound on the delay for KafkaProducer batching in seconds (floating # point value) #producer_batch_timeout = 0.0 # Size of batch for the producer async send (integer value) #producer_batch_size = 16384 [oslo_messaging_notifications] # From oslo.messaging driver = messaging # The Drivers(s) to handle sending notifications. Possible values are # messaging, messagingv2, routing, log, test, noop (multi valued) # Deprecated group/name - [DEFAULT]/notification_driver #driver = # A URL representing the messaging driver to use for notifications. If not set, # we fall back to the same configuration used for RPC. (string value) # Deprecated group/name - [DEFAULT]/notification_transport_url #transport_url = <None> # AMQP topic used for OpenStack notifications. (list value) # Deprecated group/name - [rpc_notifier2]/topics # Deprecated group/name - [DEFAULT]/notification_topics #topics = notifications # The maximum number of attempts to re-send a notification message which failed # to be delivered due to a recoverable error. 0 - No retry, -1 - indefinite # (integer value) #retry = -1 [oslo_messaging_rabbit] # From oslo.messaging # Use durable queues in AMQP. (boolean value) # Deprecated group/name - [DEFAULT]/amqp_durable_queues # Deprecated group/name - [DEFAULT]/rabbit_durable_queues #amqp_durable_queues = false # Auto-delete queues in AMQP. (boolean value) #amqp_auto_delete = false # Enable SSL (boolean value) #ssl = <None> # SSL version to use (valid only if SSL enabled). Valid values are TLSv1 and # SSLv23. SSLv2, SSLv3, TLSv1_1, and TLSv1_2 may be available on some # distributions. (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_version #ssl_version = # SSL key file (valid only if SSL enabled). (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_keyfile #ssl_key_file = # SSL cert file (valid only if SSL enabled). (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_certfile #ssl_cert_file = # SSL certification authority file (valid only if SSL enabled). (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_ca_certs #ssl_ca_file = # How long to wait before reconnecting in response to an AMQP consumer cancel # notification. (floating point value) #kombu_reconnect_delay = 1.0 # EXPERIMENTAL: Possible values are: gzip, bz2. If not set compression will not # be used. This option may not be available in future versions. (string value) #kombu_compression = <None> # How long to wait a missing client before abandoning to send it its replies. # This value should not be longer than rpc_response_timeout. (integer value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_reconnect_timeout #kombu_missing_consumer_retry_timeout = 60 # Determines how the next RabbitMQ node is chosen in case the one we are # currently connected to becomes unavailable. Takes effect only if more than # one RabbitMQ node is provided in config. (string value) # Possible values: # round-robin - <No description provided> # shuffle - <No description provided> #kombu_failover_strategy = round-robin # DEPRECATED: The RabbitMQ broker address where a single node is used. (string # value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_host = localhost # DEPRECATED: The RabbitMQ broker port where a single node is used. (port # value) # Minimum value: 0 # Maximum value: 65535 # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_port = 5672 # DEPRECATED: RabbitMQ HA cluster host:port pairs. (list value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_hosts = $rabbit_host:$rabbit_port # DEPRECATED: The RabbitMQ userid. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_userid = guest # DEPRECATED: The RabbitMQ password. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_password = guest # The RabbitMQ login method. (string value) # Possible values: # PLAIN - <No description provided> # AMQPLAIN - <No description provided> # RABBIT-CR-DEMO - <No description provided> #rabbit_login_method = AMQPLAIN # DEPRECATED: The RabbitMQ virtual host. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_virtual_host = / # How frequently to retry connecting with RabbitMQ. (integer value) #rabbit_retry_interval = 1 # How long to backoff for between retries when connecting to RabbitMQ. (integer # value) #rabbit_retry_backoff = 2 # Maximum interval of RabbitMQ connection retries. Default is 30 seconds. # (integer value) #rabbit_interval_max = 30 # DEPRECATED: Maximum number of RabbitMQ connection retries. Default is 0 # (infinite retry count). (integer value) # This option is deprecated for removal. # Its value may be silently ignored in the future. #rabbit_max_retries = 0 # Try to use HA queues in RabbitMQ (x-ha-policy: all). If you change this # option, you must wipe the RabbitMQ database. In RabbitMQ 3.0, queue mirroring # is no longer controlled by the x-ha-policy argument when declaring a queue. # If you just want to make sure that all queues (except those with auto- # generated names) are mirrored across all nodes, run: "rabbitmqctl set_policy # HA '^(?!amq\.).*' '{"ha-mode": "all"}' " (boolean value) #rabbit_ha_queues = false # Positive integer representing duration in seconds for queue TTL (x-expires). # Queues which are unused for the duration of the TTL are automatically # deleted. The parameter affects only reply and fanout queues. (integer value) # Minimum value: 1 #rabbit_transient_queues_ttl = 1800 # Specifies the number of messages to prefetch. Setting to zero allows # unlimited messages. (integer value) #rabbit_qos_prefetch_count = 0 # Number of seconds after which the Rabbit broker is considered down if # heartbeat's keep-alive fails (0 disable the heartbeat). EXPERIMENTAL (integer # value) #heartbeat_timeout_threshold = 60 # How often times during the heartbeat_timeout_threshold we check the # heartbeat. (integer value) #heartbeat_rate = 2 # Deprecated, use rpc_backend=kombu+memory or rpc_backend=fake (boolean value) #fake_rabbit = false # Maximum number of channels to allow (integer value) #channel_max = <None> # The maximum byte size for an AMQP frame (integer value) #frame_max = <None> # How often to send heartbeats for consumer's connections (integer value) #heartbeat_interval = 3 # Arguments passed to ssl.wrap_socket (dict value) #ssl_options = <None> # Set socket timeout in seconds for connection's socket (floating point value) #socket_timeout = 0.25 # Set TCP_USER_TIMEOUT in seconds for connection's socket (floating point # value) #tcp_user_timeout = 0.25 # Set delay for reconnection to some host which has connection error (floating # point value) #host_connection_reconnect_delay = 0.25 # Connection factory implementation (string value) # Possible values: # new - <No description provided> # single - <No description provided> # read_write - <No description provided> #connection_factory = single # Maximum number of connections to keep queued. (integer value) #pool_max_size = 30 # Maximum number of connections to create above `pool_max_size`. (integer # value) #pool_max_overflow = 0 # Default number of seconds to wait for a connections to available (integer # value) #pool_timeout = 30 # Lifetime of a connection (since creation) in seconds or None for no # recycling. Expired connections are closed on acquire. (integer value) #pool_recycle = 600 # Threshold at which inactive (since release) connections are considered stale # in seconds or None for no staleness. Stale connections are closed on acquire. # (integer value) #pool_stale = 60 # Default serialization mechanism for serializing/deserializing # outgoing/incoming messages (string value) # Possible values: # json - <No description provided> # msgpack - <No description provided> #default_serializer_type = json # Persist notification messages. (boolean value) #notification_persistence = false # Exchange name for sending notifications (string value) #default_notification_exchange = ${control_exchange}_notification # Max number of not acknowledged message which RabbitMQ can send to # notification listener. (integer value) #notification_listener_prefetch_count = 100 # Reconnecting retry count in case of connectivity problem during sending # notification, -1 means infinite retry. (integer value) #default_notification_retry_attempts = -1 # Reconnecting retry delay in case of connectivity problem during sending # notification message (floating point value) #notification_retry_delay = 0.25 # Time to live for rpc queues without consumers in seconds. (integer value) #rpc_queue_expiration = 60 # Exchange name for sending RPC messages (string value) #default_rpc_exchange = ${control_exchange}_rpc # Exchange name for receiving RPC replies (string value) #rpc_reply_exchange = ${control_exchange}_rpc_reply # Max number of not acknowledged message which RabbitMQ can send to rpc # listener. (integer value) #rpc_listener_prefetch_count = 100 # Max number of not acknowledged message which RabbitMQ can send to rpc reply # listener. (integer value) #rpc_reply_listener_prefetch_count = 100 # Reconnecting retry count in case of connectivity problem during sending # reply. -1 means infinite retry during rpc_timeout (integer value) #rpc_reply_retry_attempts = -1 # Reconnecting retry delay in case of connectivity problem during sending # reply. (floating point value) #rpc_reply_retry_delay = 0.25 # Reconnecting retry count in case of connectivity problem during sending RPC # message, -1 means infinite retry. If actual retry attempts in not 0 the rpc # request could be processed more than one time (integer value) #default_rpc_retry_attempts = -1 # Reconnecting retry delay in case of connectivity problem during sending RPC # message (floating point value) #rpc_retry_delay = 0.25 [oslo_messaging_zmq] # From oslo.messaging # ZeroMQ bind address. Should be a wildcard (*), an ethernet interface, or IP. # The "host" option should point or resolve to this address. (string value) #rpc_zmq_bind_address = * # MatchMaker driver. (string value) # Possible values: # redis - <No description provided> # sentinel - <No description provided> # dummy - <No description provided> #rpc_zmq_matchmaker = redis # Number of ZeroMQ contexts, defaults to 1. (integer value) #rpc_zmq_contexts = 1 # Maximum number of ingress messages to locally buffer per topic. Default is # unlimited. (integer value) #rpc_zmq_topic_backlog = <None> # Directory for holding IPC sockets. (string value) #rpc_zmq_ipc_dir = /var/run/openstack # Name of this node. Must be a valid hostname, FQDN, or IP address. Must match # "host" option, if running Nova. (string value) #rpc_zmq_host = localhost # Number of seconds to wait before all pending messages will be sent after # closing a socket. The default value of -1 specifies an infinite linger # period. The value of 0 specifies no linger period. Pending messages shall be # discarded immediately when the socket is closed. Positive values specify an # upper bound for the linger period. (integer value) # Deprecated group/name - [DEFAULT]/rpc_cast_timeout #zmq_linger = -1 # The default number of seconds that poll should wait. Poll raises timeout # exception when timeout expired. (integer value) #rpc_poll_timeout = 1 # Expiration timeout in seconds of a name service record about existing target # ( < 0 means no timeout). (integer value) #zmq_target_expire = 300 # Update period in seconds of a name service record about existing target. # (integer value) #zmq_target_update = 180 # Use PUB/SUB pattern for fanout methods. PUB/SUB always uses proxy. (boolean # value) #use_pub_sub = false # Use ROUTER remote proxy. (boolean value) #use_router_proxy = false # This option makes direct connections dynamic or static. It makes sense only # with use_router_proxy=False which means to use direct connections for direct # message types (ignored otherwise). (boolean value) #use_dynamic_connections = false # How many additional connections to a host will be made for failover reasons. # This option is actual only in dynamic connections mode. (integer value) #zmq_failover_connections = 2 # Minimal port number for random ports range. (port value) # Minimum value: 0 # Maximum value: 65535 #rpc_zmq_min_port = 49153 # Maximal port number for random ports range. (integer value) # Minimum value: 1 # Maximum value: 65536 #rpc_zmq_max_port = 65536 # Number of retries to find free port number before fail with ZMQBindError. # (integer value) #rpc_zmq_bind_port_retries = 100 # Default serialization mechanism for serializing/deserializing # outgoing/incoming messages (string value) # Possible values: # json - <No description provided> # msgpack - <No description provided> #rpc_zmq_serialization = json # This option configures round-robin mode in zmq socket. True means not keeping # a queue when server side disconnects. False means to keep queue and messages # even if server is disconnected, when the server appears we send all # accumulated messages to it. (boolean value) #zmq_immediate = true # Enable/disable TCP keepalive (KA) mechanism. The default value of -1 (or any # other negative value) means to skip any overrides and leave it to OS default; # 0 and 1 (or any other positive value) mean to disable and enable the option # respectively. (integer value) #zmq_tcp_keepalive = -1 # The duration between two keepalive transmissions in idle condition. The unit # is platform dependent, for example, seconds in Linux, milliseconds in Windows # etc. The default value of -1 (or any other negative value and 0) means to # skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_idle = -1 # The number of retransmissions to be carried out before declaring that remote # end is not available. The default value of -1 (or any other negative value # and 0) means to skip any overrides and leave it to OS default. (integer # value) #zmq_tcp_keepalive_cnt = -1 # The duration between two successive keepalive retransmissions, if # acknowledgement to the previous keepalive transmission is not received. The # unit is platform dependent, for example, seconds in Linux, milliseconds in # Windows etc. The default value of -1 (or any other negative value and 0) # means to skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_intvl = -1 # Maximum number of (green) threads to work concurrently. (integer value) #rpc_thread_pool_size = 100 # Expiration timeout in seconds of a sent/received message after which it is # not tracked anymore by a client/server. (integer value) #rpc_message_ttl = 300 # Wait for message acknowledgements from receivers. This mechanism works only # via proxy without PUB/SUB. (boolean value) #rpc_use_acks = false # Number of seconds to wait for an ack from a cast/call. After each retry # attempt this timeout is multiplied by some specified multiplier. (integer # value) #rpc_ack_timeout_base = 15 # Number to multiply base ack timeout by after each retry attempt. (integer # value) #rpc_ack_timeout_multiplier = 2 # Default number of message sending attempts in case of any problems occurred: # positive value N means at most N retries, 0 means no retries, None or -1 (or # any other negative values) mean to retry forever. This option is used only if # acknowledgments are enabled. (integer value) #rpc_retry_attempts = 3 # List of publisher hosts SubConsumer can subscribe on. This option has higher # priority then the default publishers list taken from the matchmaker. (list # value) #subscribe_on = [oslo_policy] # From oslo.policy # This option controls whether or not to enforce scope when evaluating # policies. If ``True``, the scope of the token used in the request is compared # to the ``scope_types`` of the policy being enforced. If the scopes do not # match, an ``InvalidScope`` exception will be raised. If ``False``, a message # will be logged informing operators that policies are being invoked with # mismatching scope. (boolean value) #enforce_scope = false # The file that defines policies. (string value) #policy_file = policy.json # Default rule. Enforced when a requested rule is not found. (string value) #policy_default_rule = default # Directories where policy configuration files are stored. They can be relative # to any directory in the search path defined by the config_dir option, or # absolute paths. The file defined by policy_file must exist for these # directories to be searched. Missing or empty directories are ignored. (multi # valued) #policy_dirs = policy.d # Content Type to send and receive data for REST based policy check (string # value) # Possible values: # application/x-www-form-urlencoded - <No description provided> # application/json - <No description provided> #remote_content_type = application/x-www-form-urlencoded # server identity verification for REST based policy check (boolean value) #remote_ssl_verify_server_crt = false # Absolute path to ca cert file for REST based policy check (string value) #remote_ssl_ca_crt_file = <None> # Absolute path to client cert for REST based policy check (string value) #remote_ssl_client_crt_file = <None> # Absolute path client key file REST based policy check (string value) #remote_ssl_client_key_file = <None> [pci] # From zun.conf # An alias for a PCI passthrough device requirement. # Possible Values: # * A list of JSON values which describe the aliases. For example: # alias = { # "name": "QuickAssist", # "product_id": "0443", # "vendor_id": "8086", # "device_type": "PCI" # } # defines an alias for the Intel QuickAssist card. (multi valued). Valid key # values are : # * "name": Name of the PCI alias. # * "product_id": Product ID of the device in hexadecimal. # * "vendor_id": Vendor ID of the device in hexadecimal. # * "device_type": Type of PCI device. Valid values are: "type-PCI", # "PF" and "VF". # (multi valued) #alias = # White list of PCI devices available to VMs. # Possible values: # * A JSON dictionary which describe a whitelisted PCI device. It should take # the following format: # ["vendor_id": "<id>",] ["product_id": "<id>",] # ["address": "[[[[<domain>]:]<bus>]:][<slot>][.[<function>]]" | # "devname": "<name>",] # {"<tag>": "<tag_value>",} # Where '[' indicates zero or one occurrences, '{' indicates zero or multiple # occurrences, and '|' mutually exclusive options. Note that any missing # fields are automatically wildcarded. # Valid key values are : # * "vendor_id": Vendor ID of the device in hexadecimal. # * "product_id": Product ID of the device in hexadecimal. # * "address": PCI address of the device. # * "devname": Device name of the device (for e.g. interface name). Not all # PCI devices have a name. # * "<tag>": Additional <tag> and <tag_value> used for matching PCI devices. # Supported <tag>: "physical_network". # The address key supports traditional glob style and regular expression # syntax. Valid examples are: # passthrough_whitelist = {"devname":"eth0", # "physical_network":"physnet"} # passthrough_whitelist = {"address":"*:0a:00.*"} # passthrough_whitelist = {"address":":0a:00.", # "physical_network":"physnet1"} # passthrough_whitelist = {"vendor_id":"1137", # "product_id":"0071"} # passthrough_whitelist = {"vendor_id":"1137", # "product_id":"0071", # "address": "0000:0a:00.1", # "physical_network":"physnet1"} # passthrough_whitelist = {"address":{"domain": ".*", # "bus": "02", "slot": "01", # "function": "[2-7]"}, # "physical_network":"physnet1"} # passthrough_whitelist = {"address":{"domain": ".*", # "bus": "02", "slot": "0[1-2]", # "function": ".*"}, # "physical_network":"physnet1"} # The following are invalid, as they specify mutually exclusive options: # passthrough_whitelist = {"devname":"eth0", # "physical_network":"physnet", # "address":"*:0a:00.*"} # * A JSON list of JSON dictionaries corresponding to the above format. For # example: # passthrough_whitelist = [{"product_id":"0001", "vendor_id":"8086"}, # {"product_id":"0002", "vendor_id":"8086"}] # (multi valued) #passthrough_whitelist = [profiler] # OSprofiler library allows to trace requests going through various OpenStack # services and create the accumulated report of what time was spent on each # request processing step. # From zun.conf # Enables the profiling for all services on this node. Default value is False # (fully disable the profiling feature). # Possible values: # * True: Enables the feature # * False: Disables the feature. The profiling cannot be started via this # project # operations. If the profiling is triggered by another project, this project # part # will be empty. # (boolean value) # Deprecated group/name - [profiler]/profiler_enabled #enabled = false # Enables SQL requests profiling in services. Default value is False (SQL # requests won't be traced). # Possible values: # * True: Enables SQL requests profiling. Each SQL query will be part of the # trace and can the be analyzed by how much time was spent for that. # * False: Disables SQL requests profiling. The spent time is only shown on a # higher level of operations. Single SQL queries cannot be analyzed this # way. # (boolean value) #trace_sqlalchemy = false # Secret key(s) to use for encrypting context data for performance profiling. # This string value should have the following format: # <key1>[,<key2>,...<keyn>], # where each key is some random string. A user who triggers the profiling via # the REST API has to set one of these keys in the headers of the REST API call # to include profiling results of this node for this particular project. # Both "enabled" flag and "hmac_keys" config options should be set to enable # profiling. Also, to generate correct profiling information across all # services # at least one key needs to be consistent between OpenStack projects. This # ensures it can be used from client side to generate the trace, containing # information from all possible resources. (string value) #hmac_keys = SECRET_KEY # Connection string for a notifier backend. Default value is messaging:// which # sets the notifier to oslo_messaging. # Examples of possible values: # * messaging://: use oslo_messaging driver for sending notifications. # * mongodb://127.0.0.1:27017 : use mongodb driver for sending notifications. # * elasticsearch://127.0.0.1:9200 : use elasticsearch driver for sending # notifications. # (string value) #connection_string = messaging:// # Document type for notification indexing in elasticsearch. # (string value) #es_doc_type = notification # This parameter is a time value parameter (for example: es_scroll_time=2m), # indicating for how long the nodes that participate in the search will # maintain # relevant resources in order to continue and support it. # (string value) #es_scroll_time = 2m # Elasticsearch splits large requests in batches. This parameter defines # maximum size of each batch (for example: es_scroll_size=10000). # (integer value) #es_scroll_size = 10000 # Redissentinel provides a timeout option on the connections. # This parameter defines that timeout (for example: socket_timeout=0.1). # (floating point value) #socket_timeout = 0.1 # Redissentinel uses a service name to identify a master redis service. # This parameter defines the name (for example: # sentinal_service_name=mymaster). # (string value) #sentinel_service_name = mymaster [scheduler] # From zun.conf # The class of the driver used by the scheduler. # The options are chosen from the entry points under the namespace # 'zun.scheduler.driver' in 'setup.cfg'. # Possible values: # * A string, where the string corresponds to the class name of a scheduler # driver. There are a number of options available: # ** 'chance_scheduler', which simply picks a host at random # ** A custom scheduler driver. In this case, you will be responsible for # creating and maintaining the entry point in your 'setup.cfg' file # (string value) # Possible values: # chance_scheduler - <No description provided> # fake_scheduler - <No description provided> # filter_scheduler - <No description provided> #driver = filter_scheduler # Filters that the scheduler can use. # An unordered list of the filter classes the zun scheduler may apply. Only # the # filters specified in the 'scheduler_enabled_filters' option will be used, but # any filter appearing in that option must also be included in this list. # By default, this is set to all filters that are included with zun. # This option is only used by the FilterScheduler and its subclasses; if you # use # a different scheduler, this option has no effect. # Possible values: # * A list of zero or more strings, where each string corresponds to the name # a filter that may be used for selecting a host # Related options: # * scheduler_enabled_filters # (multi valued) #available_filters = zun.scheduler.filters.all_filters # Filters that the scheduler will use. # An ordered list of filter class names that will be used for filtering # hosts. Ignore the word 'default' in the name of this option: these filters # will # *always* be applied, and they will be applied in the order they are listed so # place your most restrictive filters first to make the filtering process more # efficient. # This option is only used by the FilterScheduler and its subclasses; if you # use # a different scheduler, this option has no effect. # Possible values: # * A list of zero or more strings, where each string corresponds to the name # a filter to be used for selecting a host # Related options: # * All of the filters in this option *must* be present in the # 'scheduler_available_filters' option, or a SchedulerHostFilterNotFound # exception will be raised. # (list value) #enabled_filters = CPUFilter,RamFilter,ComputeFilter [ssl] # From zun.conf # CA certificate file to use to verify connecting clients. (string value) # Deprecated group/name - [DEFAULT]/ssl_ca_file #ca_file = <None> # Certificate file to use when starting the server securely. (string value) # Deprecated group/name - [DEFAULT]/ssl_cert_file #cert_file = <None> # Private key file to use when starting the server securely. (string value) # Deprecated group/name - [DEFAULT]/ssl_key_file #key_file = <None> # SSL version to use (valid only if SSL enabled). Valid values are TLSv1 and # SSLv23. SSLv2, SSLv3, TLSv1_1, and TLSv1_2 may be available on some # distributions. (string value) #version = <None> # Sets the list of available ciphers. value should be a string in the OpenSSL # cipher list format. (string value) #ciphers = <None> [volume] # From zun.conf # Defines which driver to use for container volume. (string value) #driver = cinder # At which the docker volume will create. (string value) #volume_dir = $state_path/mnt # Default filesystem type for volume. (string value) #fstype = ext4 # Use multipath connection of volume # Volumes can be connected as multipath devices. This will provide high # availability and fault tolerance. # (boolean value) #use_multipath = false [websocket_proxy] # Users use the websocket proxy to connect to containers, instead of # connecting to containers directly, hence protects the socket daemon. # From zun.conf wsproxy_host = 192.168.0.1 wsproxy_port = 6784 # The URL an end user would use to connect to the ``zun-wsproxy`` service. # The ``zun-wsproxy`` service is called with this token enriched URL # and establishes the connection to the proper instance. # Related options: # * The IP address must be the same as the address to which the # ``zun-wsproxy`` service is listening (see option ``wsproxy_host`` # in this section). # * The port must be the same as ``wsproxy_port``in this section. # (uri value) #base_url = ws://$wsproxy_host:$wsproxy_port/ # The IP address which is used by the ``zun-wsproxy`` service to listen # for incoming requests. # The ``zun-wsproxy`` service listens on this IP address for incoming # connection requests. # Related options: # * Ensure that this is the same IP address which is defined in the option # ``base_url`` of this section or use ``0.0.0.0`` to listen on all addresses. # (string value) #wsproxy_host = $my_ip # The port number which is used by the ``zun-wsproxy`` service to listen # for incoming requests. # The ``zun-wsproxy`` service listens on this port number for incoming # connection requests. # Related options: # * Ensure that this is the same port number as that defined in the option # ``base_url`` of this section. # (port value) # Minimum value: 0 # Maximum value: 65535 #wsproxy_port = 6784 # Adds list of allowed origins to the console websocket proxy to allow # connections from other origin hostnames. # Websocket proxy matches the host header with the origin header to # prevent cross-site requests. This list specifies if any there are # values other than host are allowed in the origin header. # Possible values: # * A list where each element is an allowed origin hostnames, else an empty # list # (list value) #allowed_origins = # Disallow non-encrypted connections. (boolean value) #ssl_only = <None> # Path to SSL certificate file. (string value) #cert = # SSL key file (if separate from cert). (string value) #key = [zun_client] # From zun.conf # Region in Identity service catalog to use for communication with the # OpenStack service. (string value) #region_name = <None> # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false _________________________________________________________________________________ # cat /etc/zun/zun.conf #THIS THE COMPUTE [DEFAULT] # From oslo.log transport_url = rabbit://openstack:ce4e1a259c263b65d4d3@controller state_path = /var/lib/zun # If set to true, the logging level will be set to DEBUG instead of the default # INFO level. (boolean value) # Note: This option can be changed without restarting. #debug = false # The name of a logging configuration file. This file is appended to any # existing logging configuration files. For details about logging configuration # files, see the Python logging module documentation. Note that when logging # configuration files are used then all logging configuration is set in the # configuration file and other logging configuration options are ignored (for # example, logging_context_format_string). (string value) # Note: This option can be changed without restarting. # Deprecated group/name - [DEFAULT]/log_config #log_config_append = <None> # Defines the format string for %%(asctime)s in log records. Default: # %(default)s . This option is ignored if log_config_append is set. (string # value) #log_date_format = %Y-%m-%d %H:%M:%S # (Optional) Name of log file to send logging output to. If no default is set, # logging will go to stderr as defined by use_stderr. This option is ignored if # log_config_append is set. (string value) # Deprecated group/name - [DEFAULT]/logfile #log_file = <None> # (Optional) The base directory used for relative log_file paths. This option # is ignored if log_config_append is set. (string value) # Deprecated group/name - [DEFAULT]/logdir #log_dir = <None> # Uses logging handler designed to watch file system. When log file is moved or # removed this handler will open a new log file with specified path # instantaneously. It makes sense only if log_file option is specified and # Linux platform is used. This option is ignored if log_config_append is set. # (boolean value) #watch_log_file = false # Use syslog for logging. Existing syslog format is DEPRECATED and will be # changed later to honor RFC5424. This option is ignored if log_config_append # is set. (boolean value) #use_syslog = false # Enable journald for logging. If running in a systemd environment you may wish # to enable journal support. Doing so will use the journal native protocol # which includes structured metadata in addition to log messages.This option is # ignored if log_config_append is set. (boolean value) #use_journal = false # Syslog facility to receive log lines. This option is ignored if # log_config_append is set. (string value) #syslog_log_facility = LOG_USER # Use JSON formatting for logging. This option is ignored if log_config_append # is set. (boolean value) #use_json = false # Log output to standard error. This option is ignored if log_config_append is # set. (boolean value) #use_stderr = false # Format string to use for log messages with context. (string value) #logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s # Format string to use for log messages when context is undefined. (string # value) #logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s # Additional data to append to log message when logging level for the message # is DEBUG. (string value) #logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d # Prefix each line of exception output with this format. (string value) #logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s # Defines the format string for %(user_identity)s that is used in # logging_context_format_string. (string value) #logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s # List of package logging levels in logger=LEVEL pairs. This option is ignored # if log_config_append is set. (list value) #default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,oslo_messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN,keystoneauth=WARN,oslo.cache=INFO,dogpile.core.dogpile=INFO # Enables or disables publication of error events. (boolean value) #publish_errors = false # The format for an instance that is passed with the log message. (string # value) #instance_format = "[instance: %(uuid)s] " # The format for an instance UUID that is passed with the log message. (string # value) #instance_uuid_format = "[instance: %(uuid)s] " # Interval, number of seconds, of log rate limiting. (integer value) #rate_limit_interval = 0 # Maximum number of logged messages per rate_limit_interval. (integer value) #rate_limit_burst = 0 # Log level name used by rate limiting: CRITICAL, ERROR, INFO, WARNING, DEBUG # or empty string. Logs with level greater or equal to rate_limit_except_level # are not filtered. An empty string means that all levels are filtered. (string # value) #rate_limit_except_level = CRITICAL # Enables or disables fatal status of deprecations. (boolean value) #fatal_deprecations = false # From oslo.messaging # Size of RPC connection pool. (integer value) #rpc_conn_pool_size = 30 # The pool size limit for connections expiration policy (integer value) #conn_pool_min_size = 2 # The time-to-live in sec of idle connections in the pool (integer value) #conn_pool_ttl = 1200 # ZeroMQ bind address. Should be a wildcard (*), an ethernet interface, or IP. # The "host" option should point or resolve to this address. (string value) #rpc_zmq_bind_address = * # MatchMaker driver. (string value) # Possible values: # redis - <No description provided> # sentinel - <No description provided> # dummy - <No description provided> #rpc_zmq_matchmaker = redis # Number of ZeroMQ contexts, defaults to 1. (integer value) #rpc_zmq_contexts = 1 # Maximum number of ingress messages to locally buffer per topic. Default is # unlimited. (integer value) #rpc_zmq_topic_backlog = <None> # Directory for holding IPC sockets. (string value) #rpc_zmq_ipc_dir = /var/run/openstack # Name of this node. Must be a valid hostname, FQDN, or IP address. Must match # "host" option, if running Nova. (string value) #rpc_zmq_host = localhost # Number of seconds to wait before all pending messages will be sent after # closing a socket. The default value of -1 specifies an infinite linger # period. The value of 0 specifies no linger period. Pending messages shall be # discarded immediately when the socket is closed. Positive values specify an # upper bound for the linger period. (integer value) # Deprecated group/name - [DEFAULT]/rpc_cast_timeout #zmq_linger = -1 # The default number of seconds that poll should wait. Poll raises timeout # exception when timeout expired. (integer value) #rpc_poll_timeout = 1 # Expiration timeout in seconds of a name service record about existing target # ( < 0 means no timeout). (integer value) #zmq_target_expire = 300 # Update period in seconds of a name service record about existing target. # (integer value) #zmq_target_update = 180 # Use PUB/SUB pattern for fanout methods. PUB/SUB always uses proxy. (boolean # value) #use_pub_sub = false # Use ROUTER remote proxy. (boolean value) #use_router_proxy = false # This option makes direct connections dynamic or static. It makes sense only # with use_router_proxy=False which means to use direct connections for direct # message types (ignored otherwise). (boolean value) #use_dynamic_connections = false # How many additional connections to a host will be made for failover reasons. # This option is actual only in dynamic connections mode. (integer value) #zmq_failover_connections = 2 # Minimal port number for random ports range. (port value) # Minimum value: 0 # Maximum value: 65535 #rpc_zmq_min_port = 49153 # Maximal port number for random ports range. (integer value) # Minimum value: 1 # Maximum value: 65536 #rpc_zmq_max_port = 65536 # Number of retries to find free port number before fail with ZMQBindError. # (integer value) #rpc_zmq_bind_port_retries = 100 # Default serialization mechanism for serializing/deserializing # outgoing/incoming messages (string value) # Possible values: # json - <No description provided> # msgpack - <No description provided> #rpc_zmq_serialization = json # This option configures round-robin mode in zmq socket. True means not keeping # a queue when server side disconnects. False means to keep queue and messages # even if server is disconnected, when the server appears we send all # accumulated messages to it. (boolean value) #zmq_immediate = true # Enable/disable TCP keepalive (KA) mechanism. The default value of -1 (or any # other negative value) means to skip any overrides and leave it to OS default; # 0 and 1 (or any other positive value) mean to disable and enable the option # respectively. (integer value) #zmq_tcp_keepalive = -1 # The duration between two keepalive transmissions in idle condition. The unit # is platform dependent, for example, seconds in Linux, milliseconds in Windows # etc. The default value of -1 (or any other negative value and 0) means to # skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_idle = -1 # The number of retransmissions to be carried out before declaring that remote # end is not available. The default value of -1 (or any other negative value # and 0) means to skip any overrides and leave it to OS default. (integer # value) #zmq_tcp_keepalive_cnt = -1 # The duration between two successive keepalive retransmissions, if # acknowledgement to the previous keepalive transmission is not received. The # unit is platform dependent, for example, seconds in Linux, milliseconds in # Windows etc. The default value of -1 (or any other negative value and 0) # means to skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_intvl = -1 # Maximum number of (green) threads to work concurrently. (integer value) #rpc_thread_pool_size = 100 # Expiration timeout in seconds of a sent/received message after which it is # not tracked anymore by a client/server. (integer value) #rpc_message_ttl = 300 # Wait for message acknowledgements from receivers. This mechanism works only # via proxy without PUB/SUB. (boolean value) #rpc_use_acks = false # Number of seconds to wait for an ack from a cast/call. After each retry # attempt this timeout is multiplied by some specified multiplier. (integer # value) #rpc_ack_timeout_base = 15 # Number to multiply base ack timeout by after each retry attempt. (integer # value) #rpc_ack_timeout_multiplier = 2 # Default number of message sending attempts in case of any problems occurred: # positive value N means at most N retries, 0 means no retries, None or -1 (or # any other negative values) mean to retry forever. This option is used only if # acknowledgments are enabled. (integer value) #rpc_retry_attempts = 3 # List of publisher hosts SubConsumer can subscribe on. This option has higher # priority then the default publishers list taken from the matchmaker. (list # value) #subscribe_on = # Size of executor thread pool when executor is threading or eventlet. (integer # value) # Deprecated group/name - [DEFAULT]/rpc_thread_pool_size #executor_thread_pool_size = 64 # Seconds to wait for a response from a call. (integer value) #rpc_response_timeout = 60 # The network address and optional user credentials for connecting to the # messaging backend, in URL format. The expected format is: # driver://[user:pass@]host:port[,[userN:passN@]hostN:portN]/virtual_host?query # Example: rabbit://rabbitmq:[email protected]:5672// # For full details on the fields in the URL see the documentation of # oslo_messaging.TransportURL at # https://docs.openstack.org/oslo.messaging/latest/reference/transport.html # (string value) #transport_url = <None> # DEPRECATED: The messaging driver to use, defaults to rabbit. Other drivers # include amqp and zmq. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rpc_backend = rabbit # The default exchange under which topics are scoped. May be overridden by an # exchange name specified in the transport_url option. (string value) #control_exchange = openstack # From oslo.service.periodic_task # Some periodic tasks can be run in a separate process. Should we run them # here? (boolean value) #run_external_periodic_tasks = true # From oslo.service.service # Enable eventlet backdoor. Acceptable values are 0, <port>, and # <start>:<end>, where 0 results in listening on a random tcp port number; # <port> results in listening on the specified port number (and not enabling # backdoor if that port is in use); and <start>:<end> results in listening on # the smallest unused port number within the specified range of port numbers. # The chosen port is displayed in the service's log file. (string value) #backdoor_port = <None> # Enable eventlet backdoor, using the provided path as a unix socket that can # receive connections. This option is mutually exclusive with 'backdoor_port' # in that only one should be provided. If both are provided then the existence # of this option overrides the usage of that option. (string value) #backdoor_socket = <None> # Enables or disables logging values of all registered options when starting a # service (at DEBUG level). (boolean value) #log_options = true # Specify a timeout after which a gracefully shutdown server will exit. Zero # value means endless wait. (integer value) #graceful_shutdown_timeout = 60 # From zun.conf # Defines which driver to use for controlling container. # Possible values: # * ``docker.driver.DockerDriver`` # Services which consume this: # * ``zun-compute`` # Interdependencies to other options: # * None # (string value) #container_driver = docker.driver.DockerDriver # Time to sleep (in seconds) during waiting for an event. (integer value) #default_sleep_time = 1 # Maximum time (in seconds) to wait for an event. (integer value) #default_timeout = 600 # Define the cpusets to be excluded from pinning (string value) #floating_cpu_set = # Whether to use infra container. If set to True, # Zun will create an infra container that serves as a placeholder of a few # Linux namespaces (i.e. network namespace). Then, one or multiple containers # could join the namespaces of the infra container thus sharing resources # inside # the sandbox (i.e. the network interface). This is typically used to group # a set of high-coupled containers into a unit. If set to False, infra # container # won't be created. # (boolean value) #use_sandbox = false # Define the runtime to create container with. Default value # in Zun is ``runc``. (string value) #container_runtime = runc # The default memory swap size in MB (default is -1 which enable unlimited # swap). (integer value) #default_memory_swap = -1 # MySQL engine to use. (string value) #mysql_engine = InnoDB # Defines the list of image driver to use for downloading image. # Possible values: # * ``docker`` # * ``glance`` # Services which consume this: # * ``zun-compute`` # Interdependencies to other options: # * None # (list value) #image_driver_list = glance,docker # Container image for sandbox container. (string value) #sandbox_image = kubernetes/pause # Image driver for sandbox container. (string value) #sandbox_image_driver = docker # Image pull policy for sandbox image. (string value) #sandbox_image_pull_policy = ifnotpresent # The IP address which the host is using to connect to the management network. # Possible values: # * String with valid IP address. Default is IPv4 address of this host. # Related options: # * docker_remote_api_host # * etcd_host # * wsproxy_host # * host_ip # * my_block_storage_ip # (string value) #my_ip = <host_ipv4> # Hostname, FQDN or IP address of this host. This can be an opaque identifier. # It is not necessarily a hostname, FQDN, or IP address. However, the node name # must be valid within an AMQP key, and if using ZeroMQ, a valid hostname, # FQDN, or IP address. # Possible values: # * String with hostname, FQDN or IP address. Default is hostname of this host. # (string value) #host = <current_hostname> # The IP address which is used to connect to the block storage network. # Possible values: # * String with valid IP address. Default is IP address of this host. # Related options: # * my_ip - if my_block_storage_ip is not set, then my_ip value is used. # (string value) #my_block_storage_ip = $my_ip # Directory where the zun python module is installed. (string value) #pybasedir = /usr/local/lib/python2.7/dist-packages/zun # Directory where zun binaries are installed. (string value) #bindir = $pybasedir/bin # Top-level directory for maintaining zun's state. (string value) #state_path = $pybasedir # Max interval size between periodic tasks execution in seconds. (integer # value) #periodic_interval_max = 60 # Max interval size between periodic tasks execution in seconds. (integer # value) #service_down_time = 180 # Interval to sync container states between the database and the docker. # The interval that Zun checks the actual container state and # the state that Zun has recorded in its database. If they are inconsistent, # Zun will update the database according to the actual container state. # Possible values: # * 0: Will run at the default periodic interval. # * Any value < 0: Disables the option. # * Any positive integer in seconds. # (integer value) #sync_container_state_interval = 60 # Path to the rootwrap configuration file to use for running commands as root. # (string value) #rootwrap_config = /etc/zun/rootwrap.conf [api] # From zun.conf # The port for the zun API server. (port value) # Minimum value: 0 # Maximum value: 65535 #port = 9517 # The listen IP for the zun API server. The default is ``$my_ip``, the IP # address of this host. (IP address value) #host_ip = $my_ip # Enable the integrated stand-alone API to service requests via HTTPS instead # of HTTP. If there is a front-end service performing HTTPS offloading from the # service, this option should be False; note, you will want to change public # API endpoint to represent SSL termination URL with 'public_endpoint' option. # (boolean value) #enable_ssl_api = false # Number of workers for zun-api service. The default will be the number of CPUs # available. (integer value) #workers = <None> # The maximum number of items returned in a single response from a collection # resource. (integer value) #max_limit = 1000 # Configuration file for WSGI definition of API. (string value) #api_paste_config = api-paste.ini # Enable image validation. (boolean value) #enable_image_validation = true [cinder_client] # From zun.conf # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Version of Cinder API to use in cinderclient. (string value) #api_version = 3 # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [compute] # From zun.conf # The queue to add compute tasks to. (string value) #topic = zun-compute # Sets the scope of the check for unique container names. # The default doesn't check for unique names. If a scope for the name check is # set, a launch of a new container with a duplicate name will result in an # ''ContainerAlreadyExists'' error. The uniqueness is case-insensitive. # Setting this option can increase the usability for end users as they don't # have to distinguish among containers with the same name by their IDs. # Possible values: # * '': An empty value means that no uniqueness check is done and duplicate # names are possible. # * "project": The container name check is done only for containers within the # same project. # * "global": The container name check is done for all containers regardless of # the project. # (string value) # Possible values: # '' - <No description provided> # project - <No description provided> # global - <No description provided> #unique_container_name_scope = # restart the containers which are runningbefore the host reboots. (boolean # value) #resume_container_state = true [cors] # From oslo.middleware.cors # Indicate whether this resource may be shared with the domain received in the # requests "origin" header. Format: "<protocol>://<host>[:<port>]", no trailing # slash. Example: https://horizon.example.com (list value) #allowed_origin = <None> # Indicate that the actual request can include user credentials (boolean value) #allow_credentials = true # Indicate which headers are safe to expose to the API. Defaults to HTTP Simple # Headers. (list value) #expose_headers = # Maximum cache age of CORS preflight requests. (integer value) #max_age = 3600 # Indicate which methods can be used during the actual request. (list value) #allow_methods = OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,PATCH # Indicate which header field names may be used during the actual request. # (list value) #allow_headers = [database] # From oslo.db connection = mysql+pymysql://zun:zun@controller/zun # If True, SQLite uses synchronous mode. (boolean value) #sqlite_synchronous = true # The back end to use for the database. (string value) # Deprecated group/name - [DEFAULT]/db_backend #backend = sqlalchemy # The SQLAlchemy connection string to use to connect to the database. (string # value) # Deprecated group/name - [DEFAULT]/sql_connection # Deprecated group/name - [DATABASE]/sql_connection # Deprecated group/name - [sql]/connection #connection = <None> # The SQLAlchemy connection string to use to connect to the slave database. # (string value) #slave_connection = <None> # The SQL mode to be used for MySQL sessions. This option, including the # default, overrides any server-set SQL mode. To use whatever SQL mode is set # by the server configuration, set this to no value. Example: mysql_sql_mode= # (string value) #mysql_sql_mode = TRADITIONAL # If True, transparently enables support for handling MySQL Cluster (NDB). # (boolean value) #mysql_enable_ndb = false # Connections which have been present in the connection pool longer than this # number of seconds will be replaced with a new one the next time they are # checked out from the pool. (integer value) # Deprecated group/name - [DATABASE]/idle_timeout # Deprecated group/name - [database]/idle_timeout # Deprecated group/name - [DEFAULT]/sql_idle_timeout # Deprecated group/name - [DATABASE]/sql_idle_timeout # Deprecated group/name - [sql]/idle_timeout #connection_recycle_time = 3600 # Minimum number of SQL connections to keep open in a pool. (integer value) # Deprecated group/name - [DEFAULT]/sql_min_pool_size # Deprecated group/name - [DATABASE]/sql_min_pool_size #min_pool_size = 1 # Maximum number of SQL connections to keep open in a pool. Setting a value of # 0 indicates no limit. (integer value) # Deprecated group/name - [DEFAULT]/sql_max_pool_size # Deprecated group/name - [DATABASE]/sql_max_pool_size #max_pool_size = 5 # Maximum number of database connection retries during startup. Set to -1 to # specify an infinite retry count. (integer value) # Deprecated group/name - [DEFAULT]/sql_max_retries # Deprecated group/name - [DATABASE]/sql_max_retries #max_retries = 10 # Interval between retries of opening a SQL connection. (integer value) # Deprecated group/name - [DEFAULT]/sql_retry_interval # Deprecated group/name - [DATABASE]/reconnect_interval #retry_interval = 10 # If set, use this value for max_overflow with SQLAlchemy. (integer value) # Deprecated group/name - [DEFAULT]/sql_max_overflow # Deprecated group/name - [DATABASE]/sqlalchemy_max_overflow #max_overflow = 50 # Verbosity of SQL debugging information: 0=None, 100=Everything. (integer # value) # Minimum value: 0 # Maximum value: 100 # Deprecated group/name - [DEFAULT]/sql_connection_debug #connection_debug = 0 # Add Python stack traces to SQL as comment strings. (boolean value) # Deprecated group/name - [DEFAULT]/sql_connection_trace #connection_trace = false # If set, use this value for pool_timeout with SQLAlchemy. (integer value) # Deprecated group/name - [DATABASE]/sqlalchemy_pool_timeout #pool_timeout = <None> # Enable the experimental use of database reconnect on connection lost. # (boolean value) #use_db_reconnect = false # Seconds between retries of a database transaction. (integer value) #db_retry_interval = 1 # If True, increases the interval between retries of a database operation up to # db_max_retry_interval. (boolean value) #db_inc_retry_interval = true # If db_inc_retry_interval is set, the maximum seconds between retries of a # database operation. (integer value) #db_max_retry_interval = 10 # Maximum retries in case of connection error or deadlock error before error is # raised. Set to -1 to specify an infinite retry count. (integer value) #db_max_retries = 20 [docker] # From zun.conf # Docker remote api version. Override it according to specific docker api # version in your environment. (string value) #docker_remote_api_version = 1.26 # Default timeout in seconds for docker client operations. (integer value) #default_timeout = 60 # API endpoint of docker daemon (string value) #api_url = unix:///var/run/docker.sock # Remote API endpoint of docker daemon (string value) #docker_remote_api_url = tcp://$docker_remote_api_host:$docker_remote_api_port # If set, ignore any SSL validation issues (boolean value) #api_insecure = false # Location of CA certificates file for securing docker api requests # (tlscacert). (string value) #ca_file = <None> # Location of TLS certificate file for securing docker api requests (tlscert). # (string value) #cert_file = <None> # Location of TLS private key file for securing docker api requests (tlskey). # (string value) #key_file = <None> # Defines the remote api host for the docker daemon. (string value) #docker_remote_api_host = $my_ip # Defines the remote api port for the docker daemon. (string value) #docker_remote_api_port = 2375 # Timeout in seconds for executing a command in a docker container. (integer # value) #execute_timeout = 5 [etcd] # From zun.conf # Host IP address on which etcd service running. The default is ``$my_ip``, the # IP address of this host. (unknown value) #etcd_host = $my_ip # Port on which etcd listen client request. (port value) # Minimum value: 0 # Maximum value: 65535 #etcd_port = 2379 [glance] # From zun.conf # Shared directory where glance images located. If specified, docker will try # to load the image from the shared directory by image ID. (string value) #images_directory = $state_path/images [glance_client] # From zun.conf # Region in Identity service catalog to use for communication with the # OpenStack service. (string value) #region_name = <None> # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Version of Glance API to use in glanceclient. (string value) #api_version = 2 # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [keystone_auth] # From zun.conf memcached_servers = controller:11211 www_authenticate_uri = http://controller:5000 project_domain_name = default project_name = service user_domain_name = default password = zun username = zun auth_url = http://controller:5000 auth_type = password auth_version = v3 auth_protocol = http service_token_roles_required = True endpoint_type = internalURL # Authentication type to load (string value) # Deprecated group/name - [keystone_auth]/auth_plugin #auth_type = <None> # Config Section from which to load plugin specific options (string value) #auth_section = <None> # Authentication URL (string value) #auth_url = <None> # Scope for system operations (string value) #system_scope = <None> # Domain ID to scope to (string value) #domain_id = <None> # Domain name to scope to (string value) #domain_name = <None> # Project ID to scope to (string value) # Deprecated group/name - [keystone_auth]/tenant_id #project_id = <None> # Project name to scope to (string value) # Deprecated group/name - [keystone_auth]/tenant_name #project_name = <None> # Domain ID containing project (string value) #project_domain_id = <None> # Domain name containing project (string value) #project_domain_name = <None> # Trust ID (string value) #trust_id = <None> # Optional domain ID to use with v3 and v2 parameters. It will be used for both # the user and project domain in v3 and ignored in v2 authentication. (string # value) #default_domain_id = <None> # Optional domain name to use with v3 API and v2 parameters. It will be used # for both the user and project domain in v3 and ignored in v2 authentication. # (string value) #default_domain_name = <None> # User id (string value) #user_id = <None> # Username (string value) # Deprecated group/name - [keystone_auth]/user_name #username = <None> # User's domain id (string value) #user_domain_id = <None> # User's domain name (string value) #user_domain_name = <None> # User's password (string value) #password = <None> [keystone_authtoken] # From keystonemiddleware.auth_token memcached_servers = controller:11211 www_authenticate_uri= http://controller:5000 project_domain_name = default project_name = service user_domain_name = default password = zun username = zun auth_url = http://controller:5000 auth_type = password # Complete "public" Identity API endpoint. This endpoint should not be an # "admin" endpoint, as it should be accessible by all end users. # Unauthenticated clients are redirected to this endpoint to authenticate. # Although this endpoint should ideally be unversioned, client support in the # wild varies. If you're using a versioned v2 endpoint here, then this should # *not* be the same endpoint the service user utilizes for validating tokens, # because normal end users may not be able to reach that endpoint. (string # value) # Deprecated group/name - [keystone_authtoken]/auth_uri #www_authenticate_uri = <None> # DEPRECATED: Complete "public" Identity API endpoint. This endpoint should not # be an "admin" endpoint, as it should be accessible by all end users. # Unauthenticated clients are redirected to this endpoint to authenticate. # Although this endpoint should ideally be unversioned, client support in the # wild varies. If you're using a versioned v2 endpoint here, then this should # *not* be the same endpoint the service user utilizes for validating tokens, # because normal end users may not be able to reach that endpoint. This option # is deprecated in favor of www_authenticate_uri and will be removed in the S # release. (string value) # This option is deprecated for removal since Queens. # Its value may be silently ignored in the future. # Reason: The auth_uri option is deprecated in favor of www_authenticate_uri # and will be removed in the S release. #auth_uri = <None> # API version of the admin Identity API endpoint. (string value) #auth_version = <None> # Do not handle authorization requests within the middleware, but delegate the # authorization decision to downstream WSGI components. (boolean value) #delay_auth_decision = false # Request timeout value for communicating with Identity API server. (integer # value) #http_connect_timeout = <None> # How many times are we trying to reconnect when communicating with Identity # API Server. (integer value) #http_request_max_retries = 3 # Request environment key where the Swift cache object is stored. When # auth_token middleware is deployed with a Swift cache, use this option to have # the middleware share a caching backend with swift. Otherwise, use the # ``memcached_servers`` option instead. (string value) #cache = <None> # Required if identity server requires client certificate (string value) #certfile = <None> # Required if identity server requires client certificate (string value) #keyfile = <None> # A PEM encoded Certificate Authority to use when verifying HTTPs connections. # Defaults to system CAs. (string value) #cafile = <None> # Verify HTTPS connections. (boolean value) #insecure = false # The region in which the identity server can be found. (string value) #region_name = <None> # DEPRECATED: Directory used to cache files related to PKI tokens. This option # has been deprecated in the Ocata release and will be removed in the P # release. (string value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #signing_dir = <None> # Optionally specify a list of memcached server(s) to use for caching. If left # undefined, tokens will instead be cached in-process. (list value) # Deprecated group/name - [keystone_authtoken]/memcache_servers #memcached_servers = <None> # In order to prevent excessive effort spent validating tokens, the middleware # caches previously-seen tokens for a configurable duration (in seconds). Set # to -1 to disable caching completely. (integer value) #token_cache_time = 300 # DEPRECATED: Determines the frequency at which the list of revoked tokens is # retrieved from the Identity service (in seconds). A high number of revocation # events combined with a low cache duration may significantly reduce # performance. Only valid for PKI tokens. This option has been deprecated in # the Ocata release and will be removed in the P release. (integer value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #revocation_cache_time = 10 # (Optional) If defined, indicate whether token data should be authenticated or # authenticated and encrypted. If MAC, token data is authenticated (with HMAC) # in the cache. If ENCRYPT, token data is encrypted and authenticated in the # cache. If the value is not one of these options or empty, auth_token will # raise an exception on initialization. (string value) # Possible values: # None - <No description provided> # MAC - <No description provided> # ENCRYPT - <No description provided> #memcache_security_strategy = None # (Optional, mandatory if memcache_security_strategy is defined) This string is # used for key derivation. (string value) #memcache_secret_key = <None> # (Optional) Number of seconds memcached server is considered dead before it is # tried again. (integer value) #memcache_pool_dead_retry = 300 # (Optional) Maximum total number of open connections to every memcached # server. (integer value) #memcache_pool_maxsize = 10 # (Optional) Socket timeout in seconds for communicating with a memcached # server. (integer value) #memcache_pool_socket_timeout = 3 # (Optional) Number of seconds a connection to memcached is held unused in the # pool before it is closed. (integer value) #memcache_pool_unused_timeout = 60 # (Optional) Number of seconds that an operation will wait to get a memcached # client connection from the pool. (integer value) #memcache_pool_conn_get_timeout = 10 # (Optional) Use the advanced (eventlet safe) memcached client pool. The # advanced pool will only work under python 2.x. (boolean value) #memcache_use_advanced_pool = false # (Optional) Indicate whether to set the X-Service-Catalog header. If False, # middleware will not ask for service catalog on token validation and will not # set the X-Service-Catalog header. (boolean value) #include_service_catalog = true # Used to control the use and type of token binding. Can be set to: "disabled" # to not check token binding. "permissive" (default) to validate binding # information if the bind type is of a form known to the server and ignore it # if not. "strict" like "permissive" but if the bind type is unknown the token # will be rejected. "required" any form of token binding is needed to be # allowed. Finally the name of a binding method that must be present in tokens. # (string value) #enforce_token_bind = permissive # DEPRECATED: If true, the revocation list will be checked for cached tokens. # This requires that PKI tokens are configured on the identity server. (boolean # value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #check_revocations_for_cached = false # DEPRECATED: Hash algorithms to use for hashing PKI tokens. This may be a # single algorithm or multiple. The algorithms are those supported by Python # standard hashlib.new(). The hashes will be tried in the order given, so put # the preferred one first for performance. The result of the first hash will be # stored in the cache. This will typically be set to multiple values only while # migrating from a less secure algorithm to a more secure one. Once all the old # tokens are expired this option should be set to a single value for better # performance. (list value) # This option is deprecated for removal since Ocata. # Its value may be silently ignored in the future. # Reason: PKI token format is no longer supported. #hash_algorithms = md5 # A choice of roles that must be present in a service token. Service tokens are # allowed to request that an expired token can be used and so this check should # tightly control that only actual services should be sending this token. Roles # here are applied as an ANY check so any role in this list must be present. # For backwards compatibility reasons this currently only affects the # allow_expired check. (list value) #service_token_roles = service # For backwards compatibility reasons we must let valid service tokens pass # that don't pass the service_token_roles check as valid. Setting this true # will become the default in a future release and should be enabled if # possible. (boolean value) #service_token_roles_required = false # Prefix to prepend at the beginning of the path. Deprecated, use identity_uri. # (string value) #auth_admin_prefix = # Host providing the admin Identity API endpoint. Deprecated, use identity_uri. # (string value) #auth_host = 127.0.0.1 # Port of the admin Identity API endpoint. Deprecated, use identity_uri. # (integer value) #auth_port = 35357 # Protocol of the admin Identity API endpoint. Deprecated, use identity_uri. # (string value) # Possible values: # http - <No description provided> # https - <No description provided> #auth_protocol = https # Complete admin Identity API endpoint. This should specify the unversioned # root endpoint e.g. https://localhost:35357/ (string value) #identity_uri = <None> # This option is deprecated and may be removed in a future release. Single # shared secret with the Keystone configuration used for bootstrapping a # Keystone installation, or otherwise bypassing the normal authentication # process. This option should not be used, use `admin_user` and # `admin_password` instead. (string value) #admin_token = <None> # Service username. (string value) #admin_user = <None> # Service user password. (string value) #admin_password = <None> # Service tenant name. (string value) #admin_tenant_name = admin # Authentication type to load (string value) # Deprecated group/name - [keystone_authtoken]/auth_plugin #auth_type = <None> # Config Section from which to load plugin specific options (string value) #auth_section = <None> [matchmaker_redis] # From oslo.messaging # DEPRECATED: Host to locate redis. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #host = 127.0.0.1 # DEPRECATED: Use this port to connect to redis host. (port value) # Minimum value: 0 # Maximum value: 65535 # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #port = 6379 # DEPRECATED: Password for Redis server (optional). (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #password = # DEPRECATED: List of Redis Sentinel hosts (fault tolerance mode), e.g., # [host:port, host1:port ... ] (list value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #sentinel_hosts = # Redis replica set name. (string value) #sentinel_group_name = oslo-messaging-zeromq # Time in ms to wait between connection attempts. (integer value) #wait_timeout = 2000 # Time in ms to wait before the transaction is killed. (integer value) #check_timeout = 20000 # Timeout in ms on blocking socket operations. (integer value) #socket_timeout = 10000 [network] # From zun.conf # Defines which driver to use for container network. (string value) #driver = kuryr # The network plugin driver name, you can find it by docker plugin list. # (string value) #driver_name = kuryr [neutron_client] # From zun.conf # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [nova_client] # From zun.conf # Version of Nova API to use in novaclient. (string value) #api_version = 2.37 # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false [oslo_concurrency] # From oslo.concurrency lock_path = /var/lib/zun/tmp # Enables or disables inter-process locks. (boolean value) #disable_process_locking = false # Directory to use for lock files. For security, the specified directory # should only be writable by the user running the processes that need locking. # Defaults to environment variable OSLO_LOCK_PATH. If OSLO_LOCK_PATH is not set # in the environment, use the Python tempfile.gettempdir function to find a # suitable location. If external locks are used, a lock path must be set. # (string value) #lock_path = /tmp [oslo_messaging_amqp] # From oslo.messaging # Name for the AMQP container. must be globally unique. Defaults to a generated # UUID (string value) #container_name = <None> # Timeout for inactive connections (in seconds) (integer value) #idle_timeout = 0 # Debug: dump AMQP frames to stdout (boolean value) #trace = false # Attempt to connect via SSL. If no other ssl-related parameters are given, it # will use the system's CA-bundle to verify the server's certificate. (boolean # value) #ssl = false # CA certificate PEM file used to verify the server's certificate (string # value) #ssl_ca_file = # Self-identifying certificate PEM file for client authentication (string # value) #ssl_cert_file = # Private key PEM file used to sign ssl_cert_file certificate (optional) # (string value) #ssl_key_file = # Password for decrypting ssl_key_file (if encrypted) (string value) #ssl_key_password = <None> # By default SSL checks that the name in the server's certificate matches the # hostname in the transport_url. In some configurations it may be preferable to # use the virtual hostname instead, for example if the server uses the Server # Name Indication TLS extension (rfc6066) to provide a certificate per virtual # host. Set ssl_verify_vhost to True if the server's SSL certificate uses the # virtual host name instead of the DNS name. (boolean value) #ssl_verify_vhost = false # DEPRECATED: Accept clients using either SSL or plain TCP (boolean value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Not applicable - not a SSL server #allow_insecure_clients = false # Space separated list of acceptable SASL mechanisms (string value) #sasl_mechanisms = # Path to directory that contains the SASL configuration (string value) #sasl_config_dir = # Name of configuration file (without .conf suffix) (string value) #sasl_config_name = # SASL realm to use if no realm present in username (string value) #sasl_default_realm = # DEPRECATED: User name for message broker authentication (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Should use configuration option transport_url to provide the # username. #username = # DEPRECATED: Password for message broker authentication (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Should use configuration option transport_url to provide the # password. #password = # Seconds to pause before attempting to re-connect. (integer value) # Minimum value: 1 #connection_retry_interval = 1 # Increase the connection_retry_interval by this many seconds after each # unsuccessful failover attempt. (integer value) # Minimum value: 0 #connection_retry_backoff = 2 # Maximum limit for connection_retry_interval + connection_retry_backoff # (integer value) # Minimum value: 1 #connection_retry_interval_max = 30 # Time to pause between re-connecting an AMQP 1.0 link that failed due to a # recoverable error. (integer value) # Minimum value: 1 #link_retry_delay = 10 # The maximum number of attempts to re-send a reply message which failed due to # a recoverable error. (integer value) # Minimum value: -1 #default_reply_retry = 0 # The deadline for an rpc reply message delivery. (integer value) # Minimum value: 5 #default_reply_timeout = 30 # The deadline for an rpc cast or call message delivery. Only used when caller # does not provide a timeout expiry. (integer value) # Minimum value: 5 #default_send_timeout = 30 # The deadline for a sent notification message delivery. Only used when caller # does not provide a timeout expiry. (integer value) # Minimum value: 5 #default_notify_timeout = 30 # The duration to schedule a purge of idle sender links. Detach link after # expiry. (integer value) # Minimum value: 1 #default_sender_link_timeout = 600 # Indicates the addressing mode used by the driver. # Permitted values: # 'legacy' - use legacy non-routable addressing # 'routable' - use routable addresses # 'dynamic' - use legacy addresses if the message bus does not support routing # otherwise use routable addressing (string value) #addressing_mode = dynamic # Enable virtual host support for those message buses that do not natively # support virtual hosting (such as qpidd). When set to true the virtual host # name will be added to all message bus addresses, effectively creating a # private 'subnet' per virtual host. Set to False if the message bus supports # virtual hosting using the 'hostname' field in the AMQP 1.0 Open performative # as the name of the virtual host. (boolean value) #pseudo_vhost = true # address prefix used when sending to a specific server (string value) #server_request_prefix = exclusive # address prefix used when broadcasting to all servers (string value) #broadcast_prefix = broadcast # address prefix when sending to any server in group (string value) #group_request_prefix = unicast # Address prefix for all generated RPC addresses (string value) #rpc_address_prefix = openstack.org/om/rpc # Address prefix for all generated Notification addresses (string value) #notify_address_prefix = openstack.org/om/notify # Appended to the address prefix when sending a fanout message. Used by the # message bus to identify fanout messages. (string value) #multicast_address = multicast # Appended to the address prefix when sending to a particular RPC/Notification # server. Used by the message bus to identify messages sent to a single # destination. (string value) #unicast_address = unicast # Appended to the address prefix when sending to a group of consumers. Used by # the message bus to identify messages that should be delivered in a round- # robin fashion across consumers. (string value) #anycast_address = anycast # Exchange name used in notification addresses. # Exchange name resolution precedence: # Target.exchange if set # else default_notification_exchange if set # else control_exchange if set # else 'notify' (string value) #default_notification_exchange = <None> # Exchange name used in RPC addresses. # Exchange name resolution precedence: # Target.exchange if set # else default_rpc_exchange if set # else control_exchange if set # else 'rpc' (string value) #default_rpc_exchange = <None> # Window size for incoming RPC Reply messages. (integer value) # Minimum value: 1 #reply_link_credit = 200 # Window size for incoming RPC Request messages (integer value) # Minimum value: 1 #rpc_server_credit = 100 # Window size for incoming Notification messages (integer value) # Minimum value: 1 #notify_server_credit = 100 # Send messages of this type pre-settled. # Pre-settled messages will not receive acknowledgement # from the peer. Note well: pre-settled messages may be # silently discarded if the delivery fails. # Permitted values: # 'rpc-call' - send RPC Calls pre-settled # 'rpc-reply'- send RPC Replies pre-settled # 'rpc-cast' - Send RPC Casts pre-settled # 'notify' - Send Notifications pre-settled # (multi valued) #pre_settled = rpc-cast #pre_settled = rpc-reply [oslo_messaging_kafka] # From oslo.messaging # DEPRECATED: Default Kafka broker Host (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #kafka_default_host = localhost # DEPRECATED: Default Kafka broker Port (port value) # Minimum value: 0 # Maximum value: 65535 # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #kafka_default_port = 9092 # Max fetch bytes of Kafka consumer (integer value) #kafka_max_fetch_bytes = 1048576 # Default timeout(s) for Kafka consumers (floating point value) #kafka_consumer_timeout = 1.0 # DEPRECATED: Pool Size for Kafka Consumers (integer value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Driver no longer uses connection pool. #pool_size = 10 # DEPRECATED: The pool size limit for connections expiration policy (integer # value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Driver no longer uses connection pool. #conn_pool_min_size = 2 # DEPRECATED: The time-to-live in sec of idle connections in the pool (integer # value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Driver no longer uses connection pool. #conn_pool_ttl = 1200 # Group id for Kafka consumer. Consumers in one group will coordinate message # consumption (string value) #consumer_group = oslo_messaging_consumer # Upper bound on the delay for KafkaProducer batching in seconds (floating # point value) #producer_batch_timeout = 0.0 # Size of batch for the producer async send (integer value) #producer_batch_size = 16384 [oslo_messaging_notifications] # From oslo.messaging # The Drivers(s) to handle sending notifications. Possible values are # messaging, messagingv2, routing, log, test, noop (multi valued) # Deprecated group/name - [DEFAULT]/notification_driver #driver = # A URL representing the messaging driver to use for notifications. If not set, # we fall back to the same configuration used for RPC. (string value) # Deprecated group/name - [DEFAULT]/notification_transport_url #transport_url = <None> # AMQP topic used for OpenStack notifications. (list value) # Deprecated group/name - [rpc_notifier2]/topics # Deprecated group/name - [DEFAULT]/notification_topics #topics = notifications # The maximum number of attempts to re-send a notification message which failed # to be delivered due to a recoverable error. 0 - No retry, -1 - indefinite # (integer value) #retry = -1 [oslo_messaging_rabbit] # From oslo.messaging # Use durable queues in AMQP. (boolean value) # Deprecated group/name - [DEFAULT]/amqp_durable_queues # Deprecated group/name - [DEFAULT]/rabbit_durable_queues #amqp_durable_queues = false # Auto-delete queues in AMQP. (boolean value) #amqp_auto_delete = false # Enable SSL (boolean value) #ssl = <None> # SSL version to use (valid only if SSL enabled). Valid values are TLSv1 and # SSLv23. SSLv2, SSLv3, TLSv1_1, and TLSv1_2 may be available on some # distributions. (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_version #ssl_version = # SSL key file (valid only if SSL enabled). (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_keyfile #ssl_key_file = # SSL cert file (valid only if SSL enabled). (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_certfile #ssl_cert_file = # SSL certification authority file (valid only if SSL enabled). (string value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_ssl_ca_certs #ssl_ca_file = # How long to wait before reconnecting in response to an AMQP consumer cancel # notification. (floating point value) #kombu_reconnect_delay = 1.0 # EXPERIMENTAL: Possible values are: gzip, bz2. If not set compression will not # be used. This option may not be available in future versions. (string value) #kombu_compression = <None> # How long to wait a missing client before abandoning to send it its replies. # This value should not be longer than rpc_response_timeout. (integer value) # Deprecated group/name - [oslo_messaging_rabbit]/kombu_reconnect_timeout #kombu_missing_consumer_retry_timeout = 60 # Determines how the next RabbitMQ node is chosen in case the one we are # currently connected to becomes unavailable. Takes effect only if more than # one RabbitMQ node is provided in config. (string value) # Possible values: # round-robin - <No description provided> # shuffle - <No description provided> #kombu_failover_strategy = round-robin # DEPRECATED: The RabbitMQ broker address where a single node is used. (string # value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_host = localhost # DEPRECATED: The RabbitMQ broker port where a single node is used. (port # value) # Minimum value: 0 # Maximum value: 65535 # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_port = 5672 # DEPRECATED: RabbitMQ HA cluster host:port pairs. (list value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_hosts = $rabbit_host:$rabbit_port # DEPRECATED: The RabbitMQ userid. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_userid = guest # DEPRECATED: The RabbitMQ password. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_password = guest # The RabbitMQ login method. (string value) # Possible values: # PLAIN - <No description provided> # AMQPLAIN - <No description provided> # RABBIT-CR-DEMO - <No description provided> #rabbit_login_method = AMQPLAIN # DEPRECATED: The RabbitMQ virtual host. (string value) # This option is deprecated for removal. # Its value may be silently ignored in the future. # Reason: Replaced by [DEFAULT]/transport_url #rabbit_virtual_host = / # How frequently to retry connecting with RabbitMQ. (integer value) #rabbit_retry_interval = 1 # How long to backoff for between retries when connecting to RabbitMQ. (integer # value) #rabbit_retry_backoff = 2 # Maximum interval of RabbitMQ connection retries. Default is 30 seconds. # (integer value) #rabbit_interval_max = 30 # DEPRECATED: Maximum number of RabbitMQ connection retries. Default is 0 # (infinite retry count). (integer value) # This option is deprecated for removal. # Its value may be silently ignored in the future. #rabbit_max_retries = 0 # Try to use HA queues in RabbitMQ (x-ha-policy: all). If you change this # option, you must wipe the RabbitMQ database. In RabbitMQ 3.0, queue mirroring # is no longer controlled by the x-ha-policy argument when declaring a queue. # If you just want to make sure that all queues (except those with auto- # generated names) are mirrored across all nodes, run: "rabbitmqctl set_policy # HA '^(?!amq\.).*' '{"ha-mode": "all"}' " (boolean value) #rabbit_ha_queues = false # Positive integer representing duration in seconds for queue TTL (x-expires). # Queues which are unused for the duration of the TTL are automatically # deleted. The parameter affects only reply and fanout queues. (integer value) # Minimum value: 1 #rabbit_transient_queues_ttl = 1800 # Specifies the number of messages to prefetch. Setting to zero allows # unlimited messages. (integer value) #rabbit_qos_prefetch_count = 0 # Number of seconds after which the Rabbit broker is considered down if # heartbeat's keep-alive fails (0 disable the heartbeat). EXPERIMENTAL (integer # value) #heartbeat_timeout_threshold = 60 # How often times during the heartbeat_timeout_threshold we check the # heartbeat. (integer value) #heartbeat_rate = 2 # Deprecated, use rpc_backend=kombu+memory or rpc_backend=fake (boolean value) #fake_rabbit = false # Maximum number of channels to allow (integer value) #channel_max = <None> # The maximum byte size for an AMQP frame (integer value) #frame_max = <None> # How often to send heartbeats for consumer's connections (integer value) #heartbeat_interval = 3 # Arguments passed to ssl.wrap_socket (dict value) #ssl_options = <None> # Set socket timeout in seconds for connection's socket (floating point value) #socket_timeout = 0.25 # Set TCP_USER_TIMEOUT in seconds for connection's socket (floating point # value) #tcp_user_timeout = 0.25 # Set delay for reconnection to some host which has connection error (floating # point value) #host_connection_reconnect_delay = 0.25 # Connection factory implementation (string value) # Possible values: # new - <No description provided> # single - <No description provided> # read_write - <No description provided> #connection_factory = single # Maximum number of connections to keep queued. (integer value) #pool_max_size = 30 # Maximum number of connections to create above `pool_max_size`. (integer # value) #pool_max_overflow = 0 # Default number of seconds to wait for a connections to available (integer # value) #pool_timeout = 30 # Lifetime of a connection (since creation) in seconds or None for no # recycling. Expired connections are closed on acquire. (integer value) #pool_recycle = 600 # Threshold at which inactive (since release) connections are considered stale # in seconds or None for no staleness. Stale connections are closed on acquire. # (integer value) #pool_stale = 60 # Default serialization mechanism for serializing/deserializing # outgoing/incoming messages (string value) # Possible values: # json - <No description provided> # msgpack - <No description provided> #default_serializer_type = json # Persist notification messages. (boolean value) #notification_persistence = false # Exchange name for sending notifications (string value) #default_notification_exchange = ${control_exchange}_notification # Max number of not acknowledged message which RabbitMQ can send to # notification listener. (integer value) #notification_listener_prefetch_count = 100 # Reconnecting retry count in case of connectivity problem during sending # notification, -1 means infinite retry. (integer value) #default_notification_retry_attempts = -1 # Reconnecting retry delay in case of connectivity problem during sending # notification message (floating point value) #notification_retry_delay = 0.25 # Time to live for rpc queues without consumers in seconds. (integer value) #rpc_queue_expiration = 60 # Exchange name for sending RPC messages (string value) #default_rpc_exchange = ${control_exchange}_rpc # Exchange name for receiving RPC replies (string value) #rpc_reply_exchange = ${control_exchange}_rpc_reply # Max number of not acknowledged message which RabbitMQ can send to rpc # listener. (integer value) #rpc_listener_prefetch_count = 100 # Max number of not acknowledged message which RabbitMQ can send to rpc reply # listener. (integer value) #rpc_reply_listener_prefetch_count = 100 # Reconnecting retry count in case of connectivity problem during sending # reply. -1 means infinite retry during rpc_timeout (integer value) #rpc_reply_retry_attempts = -1 # Reconnecting retry delay in case of connectivity problem during sending # reply. (floating point value) #rpc_reply_retry_delay = 0.25 # Reconnecting retry count in case of connectivity problem during sending RPC # message, -1 means infinite retry. If actual retry attempts in not 0 the rpc # request could be processed more than one time (integer value) #default_rpc_retry_attempts = -1 # Reconnecting retry delay in case of connectivity problem during sending RPC # message (floating point value) #rpc_retry_delay = 0.25 [oslo_messaging_zmq] # From oslo.messaging # ZeroMQ bind address. Should be a wildcard (*), an ethernet interface, or IP. # The "host" option should point or resolve to this address. (string value) #rpc_zmq_bind_address = * # MatchMaker driver. (string value) # Possible values: # redis - <No description provided> # sentinel - <No description provided> # dummy - <No description provided> #rpc_zmq_matchmaker = redis # Number of ZeroMQ contexts, defaults to 1. (integer value) #rpc_zmq_contexts = 1 # Maximum number of ingress messages to locally buffer per topic. Default is # unlimited. (integer value) #rpc_zmq_topic_backlog = <None> # Directory for holding IPC sockets. (string value) #rpc_zmq_ipc_dir = /var/run/openstack # Name of this node. Must be a valid hostname, FQDN, or IP address. Must match # "host" option, if running Nova. (string value) #rpc_zmq_host = localhost # Number of seconds to wait before all pending messages will be sent after # closing a socket. The default value of -1 specifies an infinite linger # period. The value of 0 specifies no linger period. Pending messages shall be # discarded immediately when the socket is closed. Positive values specify an # upper bound for the linger period. (integer value) # Deprecated group/name - [DEFAULT]/rpc_cast_timeout #zmq_linger = -1 # The default number of seconds that poll should wait. Poll raises timeout # exception when timeout expired. (integer value) #rpc_poll_timeout = 1 # Expiration timeout in seconds of a name service record about existing target # ( < 0 means no timeout). (integer value) #zmq_target_expire = 300 # Update period in seconds of a name service record about existing target. # (integer value) #zmq_target_update = 180 # Use PUB/SUB pattern for fanout methods. PUB/SUB always uses proxy. (boolean # value) #use_pub_sub = false # Use ROUTER remote proxy. (boolean value) #use_router_proxy = false # This option makes direct connections dynamic or static. It makes sense only # with use_router_proxy=False which means to use direct connections for direct # message types (ignored otherwise). (boolean value) #use_dynamic_connections = false # How many additional connections to a host will be made for failover reasons. # This option is actual only in dynamic connections mode. (integer value) #zmq_failover_connections = 2 # Minimal port number for random ports range. (port value) # Minimum value: 0 # Maximum value: 65535 #rpc_zmq_min_port = 49153 # Maximal port number for random ports range. (integer value) # Minimum value: 1 # Maximum value: 65536 #rpc_zmq_max_port = 65536 # Number of retries to find free port number before fail with ZMQBindError. # (integer value) #rpc_zmq_bind_port_retries = 100 # Default serialization mechanism for serializing/deserializing # outgoing/incoming messages (string value) # Possible values: # json - <No description provided> # msgpack - <No description provided> #rpc_zmq_serialization = json # This option configures round-robin mode in zmq socket. True means not keeping # a queue when server side disconnects. False means to keep queue and messages # even if server is disconnected, when the server appears we send all # accumulated messages to it. (boolean value) #zmq_immediate = true # Enable/disable TCP keepalive (KA) mechanism. The default value of -1 (or any # other negative value) means to skip any overrides and leave it to OS default; # 0 and 1 (or any other positive value) mean to disable and enable the option # respectively. (integer value) #zmq_tcp_keepalive = -1 # The duration between two keepalive transmissions in idle condition. The unit # is platform dependent, for example, seconds in Linux, milliseconds in Windows # etc. The default value of -1 (or any other negative value and 0) means to # skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_idle = -1 # The number of retransmissions to be carried out before declaring that remote # end is not available. The default value of -1 (or any other negative value # and 0) means to skip any overrides and leave it to OS default. (integer # value) #zmq_tcp_keepalive_cnt = -1 # The duration between two successive keepalive retransmissions, if # acknowledgement to the previous keepalive transmission is not received. The # unit is platform dependent, for example, seconds in Linux, milliseconds in # Windows etc. The default value of -1 (or any other negative value and 0) # means to skip any overrides and leave it to OS default. (integer value) #zmq_tcp_keepalive_intvl = -1 # Maximum number of (green) threads to work concurrently. (integer value) #rpc_thread_pool_size = 100 # Expiration timeout in seconds of a sent/received message after which it is # not tracked anymore by a client/server. (integer value) #rpc_message_ttl = 300 # Wait for message acknowledgements from receivers. This mechanism works only # via proxy without PUB/SUB. (boolean value) #rpc_use_acks = false # Number of seconds to wait for an ack from a cast/call. After each retry # attempt this timeout is multiplied by some specified multiplier. (integer # value) #rpc_ack_timeout_base = 15 # Number to multiply base ack timeout by after each retry attempt. (integer # value) #rpc_ack_timeout_multiplier = 2 # Default number of message sending attempts in case of any problems occurred: # positive value N means at most N retries, 0 means no retries, None or -1 (or # any other negative values) mean to retry forever. This option is used only if # acknowledgments are enabled. (integer value) #rpc_retry_attempts = 3 # List of publisher hosts SubConsumer can subscribe on. This option has higher # priority then the default publishers list taken from the matchmaker. (list # value) #subscribe_on = [oslo_policy] # From oslo.policy # This option controls whether or not to enforce scope when evaluating # policies. If ``True``, the scope of the token used in the request is compared # to the ``scope_types`` of the policy being enforced. If the scopes do not # match, an ``InvalidScope`` exception will be raised. If ``False``, a message # will be logged informing operators that policies are being invoked with # mismatching scope. (boolean value) #enforce_scope = false # The file that defines policies. (string value) #policy_file = policy.json # Default rule. Enforced when a requested rule is not found. (string value) #policy_default_rule = default # Directories where policy configuration files are stored. They can be relative # to any directory in the search path defined by the config_dir option, or # absolute paths. The file defined by policy_file must exist for these # directories to be searched. Missing or empty directories are ignored. (multi # valued) #policy_dirs = policy.d # Content Type to send and receive data for REST based policy check (string # value) # Possible values: # application/x-www-form-urlencoded - <No description provided> # application/json - <No description provided> #remote_content_type = application/x-www-form-urlencoded # server identity verification for REST based policy check (boolean value) #remote_ssl_verify_server_crt = false # Absolute path to ca cert file for REST based policy check (string value) #remote_ssl_ca_crt_file = <None> # Absolute path to client cert for REST based policy check (string value) #remote_ssl_client_crt_file = <None> # Absolute path client key file REST based policy check (string value) #remote_ssl_client_key_file = <None> [pci] # From zun.conf # An alias for a PCI passthrough device requirement. # Possible Values: # * A list of JSON values which describe the aliases. For example: # alias = { # "name": "QuickAssist", # "product_id": "0443", # "vendor_id": "8086", # "device_type": "PCI" # } # defines an alias for the Intel QuickAssist card. (multi valued). Valid key # values are : # * "name": Name of the PCI alias. # * "product_id": Product ID of the device in hexadecimal. # * "vendor_id": Vendor ID of the device in hexadecimal. # * "device_type": Type of PCI device. Valid values are: "type-PCI", # "PF" and "VF". # (multi valued) #alias = # White list of PCI devices available to VMs. # Possible values: # * A JSON dictionary which describe a whitelisted PCI device. It should take # the following format: # ["vendor_id": "<id>",] ["product_id": "<id>",] # ["address": "[[[[<domain>]:]<bus>]:][<slot>][.[<function>]]" | # "devname": "<name>",] # {"<tag>": "<tag_value>",} # Where '[' indicates zero or one occurrences, '{' indicates zero or multiple # occurrences, and '|' mutually exclusive options. Note that any missing # fields are automatically wildcarded. # Valid key values are : # * "vendor_id": Vendor ID of the device in hexadecimal. # * "product_id": Product ID of the device in hexadecimal. # * "address": PCI address of the device. # * "devname": Device name of the device (for e.g. interface name). Not all # PCI devices have a name. # * "<tag>": Additional <tag> and <tag_value> used for matching PCI devices. # Supported <tag>: "physical_network". # The address key supports traditional glob style and regular expression # syntax. Valid examples are: # passthrough_whitelist = {"devname":"eth0", # "physical_network":"physnet"} # passthrough_whitelist = {"address":"*:0a:00.*"} # passthrough_whitelist = {"address":":0a:00.", # "physical_network":"physnet1"} # passthrough_whitelist = {"vendor_id":"1137", # "product_id":"0071"} # passthrough_whitelist = {"vendor_id":"1137", # "product_id":"0071", # "address": "0000:0a:00.1", # "physical_network":"physnet1"} # passthrough_whitelist = {"address":{"domain": ".*", # "bus": "02", "slot": "01", # "function": "[2-7]"}, # "physical_network":"physnet1"} # passthrough_whitelist = {"address":{"domain": ".*", # "bus": "02", "slot": "0[1-2]", # "function": ".*"}, # "physical_network":"physnet1"} # The following are invalid, as they specify mutually exclusive options: # passthrough_whitelist = {"devname":"eth0", # "physical_network":"physnet", # "address":"*:0a:00.*"} # * A JSON list of JSON dictionaries corresponding to the above format. For # example: # passthrough_whitelist = [{"product_id":"0001", "vendor_id":"8086"}, # {"product_id":"0002", "vendor_id":"8086"}] # (multi valued) #passthrough_whitelist = [profiler] # OSprofiler library allows to trace requests going through various OpenStack # services and create the accumulated report of what time was spent on each # request processing step. # From zun.conf # Enables the profiling for all services on this node. Default value is False # (fully disable the profiling feature). # Possible values: # * True: Enables the feature # * False: Disables the feature. The profiling cannot be started via this # project # operations. If the profiling is triggered by another project, this project # part # will be empty. # (boolean value) # Deprecated group/name - [profiler]/profiler_enabled #enabled = false # Enables SQL requests profiling in services. Default value is False (SQL # requests won't be traced). # Possible values: # * True: Enables SQL requests profiling. Each SQL query will be part of the # trace and can the be analyzed by how much time was spent for that. # * False: Disables SQL requests profiling. The spent time is only shown on a # higher level of operations. Single SQL queries cannot be analyzed this # way. # (boolean value) #trace_sqlalchemy = false # Secret key(s) to use for encrypting context data for performance profiling. # This string value should have the following format: # <key1>[,<key2>,...<keyn>], # where each key is some random string. A user who triggers the profiling via # the REST API has to set one of these keys in the headers of the REST API call # to include profiling results of this node for this particular project. # Both "enabled" flag and "hmac_keys" config options should be set to enable # profiling. Also, to generate correct profiling information across all # services # at least one key needs to be consistent between OpenStack projects. This # ensures it can be used from client side to generate the trace, containing # information from all possible resources. (string value) #hmac_keys = SECRET_KEY # Connection string for a notifier backend. Default value is messaging:// which # sets the notifier to oslo_messaging. # Examples of possible values: # * messaging://: use oslo_messaging driver for sending notifications. # * mongodb://127.0.0.1:27017 : use mongodb driver for sending notifications. # * elasticsearch://127.0.0.1:9200 : use elasticsearch driver for sending # notifications. # (string value) #connection_string = messaging:// # Document type for notification indexing in elasticsearch. # (string value) #es_doc_type = notification # This parameter is a time value parameter (for example: es_scroll_time=2m), # indicating for how long the nodes that participate in the search will # maintain # relevant resources in order to continue and support it. # (string value) #es_scroll_time = 2m # Elasticsearch splits large requests in batches. This parameter defines # maximum size of each batch (for example: es_scroll_size=10000). # (integer value) #es_scroll_size = 10000 # Redissentinel provides a timeout option on the connections. # This parameter defines that timeout (for example: socket_timeout=0.1). # (floating point value) #socket_timeout = 0.1 # Redissentinel uses a service name to identify a master redis service. # This parameter defines the name (for example: # sentinal_service_name=mymaster). # (string value) #sentinel_service_name = mymaster [scheduler] # From zun.conf # The class of the driver used by the scheduler. # The options are chosen from the entry points under the namespace # 'zun.scheduler.driver' in 'setup.cfg'. # Possible values: # * A string, where the string corresponds to the class name of a scheduler # driver. There are a number of options available: # ** 'chance_scheduler', which simply picks a host at random # ** A custom scheduler driver. In this case, you will be responsible for # creating and maintaining the entry point in your 'setup.cfg' file # (string value) # Possible values: # chance_scheduler - <No description provided> # fake_scheduler - <No description provided> # filter_scheduler - <No description provided> #driver = filter_scheduler # Filters that the scheduler can use. # An unordered list of the filter classes the zun scheduler may apply. Only # the # filters specified in the 'scheduler_enabled_filters' option will be used, but # any filter appearing in that option must also be included in this list. # By default, this is set to all filters that are included with zun. # This option is only used by the FilterScheduler and its subclasses; if you # use # a different scheduler, this option has no effect. # Possible values: # * A list of zero or more strings, where each string corresponds to the name # a filter that may be used for selecting a host # Related options: # * scheduler_enabled_filters # (multi valued) #available_filters = zun.scheduler.filters.all_filters # Filters that the scheduler will use. # An ordered list of filter class names that will be used for filtering # hosts. Ignore the word 'default' in the name of this option: these filters # will # *always* be applied, and they will be applied in the order they are listed so # place your most restrictive filters first to make the filtering process more # efficient. # This option is only used by the FilterScheduler and its subclasses; if you # use # a different scheduler, this option has no effect. # Possible values: # * A list of zero or more strings, where each string corresponds to the name # a filter to be used for selecting a host # Related options: # * All of the filters in this option *must* be present in the # 'scheduler_available_filters' option, or a SchedulerHostFilterNotFound # exception will be raised. # (list value) #enabled_filters = CPUFilter,RamFilter,ComputeFilter [ssl] # From zun.conf # CA certificate file to use to verify connecting clients. (string value) # Deprecated group/name - [DEFAULT]/ssl_ca_file #ca_file = <None> # Certificate file to use when starting the server securely. (string value) # Deprecated group/name - [DEFAULT]/ssl_cert_file #cert_file = <None> # Private key file to use when starting the server securely. (string value) # Deprecated group/name - [DEFAULT]/ssl_key_file #key_file = <None> # SSL version to use (valid only if SSL enabled). Valid values are TLSv1 and # SSLv23. SSLv2, SSLv3, TLSv1_1, and TLSv1_2 may be available on some # distributions. (string value) #version = <None> # Sets the list of available ciphers. value should be a string in the OpenSSL # cipher list format. (string value) #ciphers = <None> [volume] # From zun.conf # Defines which driver to use for container volume. (string value) #driver = cinder # At which the docker volume will create. (string value) #volume_dir = $state_path/mnt # Default filesystem type for volume. (string value) #fstype = ext4 # Use multipath connection of volume # Volumes can be connected as multipath devices. This will provide high # availability and fault tolerance. # (boolean value) #use_multipath = false [websocket_proxy] # Users use the websocket proxy to connect to containers, instead of # connecting to containers directly, hence protects the socket daemon. # From zun.conf base_url = ws://controller:6784/ # The URL an end user would use to connect to the ``zun-wsproxy`` service. # The ``zun-wsproxy`` service is called with this token enriched URL # and establishes the connection to the proper instance. # Related options: # * The IP address must be the same as the address to which the # ``zun-wsproxy`` service is listening (see option ``wsproxy_host`` # in this section). # * The port must be the same as ``wsproxy_port``in this section. # (uri value) #base_url = ws://$wsproxy_host:$wsproxy_port/ # The IP address which is used by the ``zun-wsproxy`` service to listen # for incoming requests. # The ``zun-wsproxy`` service listens on this IP address for incoming # connection requests. # Related options: # * Ensure that this is the same IP address which is defined in the option # ``base_url`` of this section or use ``0.0.0.0`` to listen on all addresses. # (string value) #wsproxy_host = $my_ip # The port number which is used by the ``zun-wsproxy`` service to listen # for incoming requests. # The ``zun-wsproxy`` service listens on this port number for incoming # connection requests. # Related options: # * Ensure that this is the same port number as that defined in the option # ``base_url`` of this section. # (port value) # Minimum value: 0 # Maximum value: 65535 #wsproxy_port = 6784 # Adds list of allowed origins to the console websocket proxy to allow # connections from other origin hostnames. # Websocket proxy matches the host header with the origin header to # prevent cross-site requests. This list specifies if any there are # values other than host are allowed in the origin header. # Possible values: # * A list where each element is an allowed origin hostnames, else an empty # list # (list value) #allowed_origins = # Disallow non-encrypted connections. (boolean value) #ssl_only = <None> # Path to SSL certificate file. (string value) #cert = # SSL key file (if separate from cert). (string value) #key = [zun_client] # From zun.conf # Region in Identity service catalog to use for communication with the # OpenStack service. (string value) #region_name = <None> # Type of endpoint in Identity service catalog to use for communication with # the OpenStack service. (string value) #endpoint_type = publicURL # Optional CA cert file to use in SSL connections. (string value) #ca_file = <None> # Optional PEM-formatted certificate chain file. (string value) #cert_file = <None> # Optional PEM-formatted file that contains the private key. (string value) #key_file = <None> # If set, then the server's certificate will not be verified. (boolean value) #insecure = false OK. First, a gentle warning that fixing this issue requires opening up Docker Daemon in the compute host, which has security implication. For details, check https://bugs.launchpad.net/zun/+bug/1735076 . In short, in stable/queens, how "exec" works is that the client send a request to server to get the URL of the docker daemon in the compute node, then access the docker daemon (which is 130.127.133.20:2375 in your case) to enter the container. It looks you cannot access the docker daemon (130.127.133.20:2375) from the host you run the command. Several things you want to check: * Whether you docker daemon is running on 130.127.133.20 and listening to port 2375 * Whether you can access the host/port (i.e. 130.127.133.20:2375) from the host you run the command. For example, you can run "telnet 172.31.2.195 2375" to check. My best guess is that there is a firewall that blocks that URL. Again, please take the security implication into account. In particular, opening up the docker daemon is a security risk. If this is a production environment and you really need the interactive exec to work, I would suggest to upgrade to stable/rocky or newer version. In a newer version, the client won't directly access the Docker Daemon (instead via a proxy that blocks unauthorized access). <div class="comment-text editable-message-text" itemprop="commentText"><p>Hi @hongbin</p> <p>the problem was resolved on the rocky release but I have an other problem:<br/> # openstack appcontainer exec --interactive container /bin/sh<br/> WARNING: Failed to import plugin clustering.<br/> connected to container, press Enter to continue<br/> type ~. to disconnect</p> <p>Internal Server Error (HTTP 500) (Request-ID: req-bba1d6e2-<wbr/>d0fe-4915-<wbr/>8522-5d0f9f12d6<wbr/>34)<br/> _______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_____</p> <p>and also i have a odoo container running but when I am trying to acces to it at 10.245.0.11:8069 or 10.245.0.11:8078 via the browser there is no response</p> <p>+------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>----+--<wbr/>-------<wbr/>--+----<wbr/>----+--<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-+<br/> | uuid | name | image | status | task_state | addresses | ports |<br/> +------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>----+--<wbr/>-------<wbr/>--+----<wbr/>----+--<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-+<br/> | fd4d5923-<wbr/>91f6-4fdd-<wbr/>aa29-5ac5d38132<wbr/>86 | odoo | odoo | Running | None | 10.254.0.11 | [8069, 8071] |<br/> | 6028d286-<wbr/>9caa-46c9-<wbr/>937e-a37f5925ea<wbr/>bf | container | cirros | Created | container_starting | 10.10.0.20 | [] |<br/> +------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>----+--<wbr/>-------<wbr/>--+----<wbr/>----+--<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-+</p></div> the problem was resolved on the rocky release but I have an other problem: # openstack appcontainer exec --interactive container /bin/sh WARNING: Failed to import plugin clustering. connected to container, press Enter to continue type ~. to disconnect Internal Server Error (HTTP 500) (Request-ID: req-bba1d6e2-d0fe-4915-8522-5d0f9f12d634) __________________________________________________________________________________ and also i have a odoo container running but when I am trying to acces to it at 10.245.0.11:8069 or 10.245.0.11:8078 via the browser there is no response +--------------------------------------+-----------+--------+---------+--------------------+-------------+--------------+ | uuid | name | image | status | task_state | addresses | ports | +--------------------------------------+-----------+--------+---------+--------------------+-------------+--------------+ | fd4d5923-91f6-4fdd-aa29-5ac5d3813286 | odoo | odoo | Running | None | 10.254.0.11 | [8069, 8071] | | 6028d286-9caa-46c9-937e-a37f5925eabf | container | cirros | Created | container_starting | 10.10.0.20 | [] | +--------------------------------------+-----------+--------+---------+--------------------+-------------+--------------+ > the problem was resolved on the rocky release but I have an other problem:<br/> > # openstack appcontainer exec --interactive container /bin/sh<br/> > WARNING: Failed to import plugin clustering.<br/> > connected to container, press Enter to continue<br/> > type ~. to disconnect<br/> > Internal Server Error (HTTP 500) (Request-ID:<br/> > req-bba1d6e2-<wbr/>d0fe-4915-<wbr/>8522-5d0f9f12d6<wbr/>34)<br/> <p>Could you add a '--debug' option and show me the output? In addition, do<br/> you find some error messages in the Zun logs (i.e. zun-api, zun-compute,<br/> zun-wsproxy).</p> <p># openstack --debug appcontainer exec --interactive container /bin/sh</p> > _______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_____<br/> > and also i have a odoo container running but when I am trying to acces<br/> > to it at 10.245.0.11:8069 or 10.245.0.11:8078 via the browser there is<br/> > no response<br/> > +------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>----+--<wbr/>-------<wbr/>--+----<wbr/>----+--<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-+<br/> > | uuid | name | image | status |<br/> > task_state | addresses | ports |<br/> > +------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>----+--<wbr/>-------<wbr/>--+----<wbr/>----+--<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-+<br/> > | fd4d5923-<wbr/>91f6-4fdd-<wbr/>aa29-5ac5d38132<wbr/>86 | odoo | odoo | Running |<br/> > None | 10.254.0.11 | [8069, 8071] |<br/> > | 6028d286-<wbr/>9caa-46c9-<wbr/>937e-a37f5925ea<wbr/>bf | container | cirros | Created |<br/> > container_starting | 10.10.0.20 | [] |<br/> > +------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>-------<wbr/>----+--<wbr/>-------<wbr/>--+----<wbr/>----+--<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>+------<wbr/>-------<wbr/>-+<br/> <p>It might relate to security groups. By default, the 'odoo' is using the<br/> 'default' security group, so you want to make sure if those ports are open<br/> in the security group. Let me know if it doesn't help.</p> > You received this bug notification because you are subscribed to Zun.<br/> > Matching subscriptions: zun<br/> > <a rel="nofollow" href="https://bugs.launchpad.net/bugs/1830450">https:/<wbr/>/bugs.launchpad<wbr/>.net/bugs/<wbr/>1830450</a><br/> > Title:<br/> > openstack appcontainer exec --interactive<br/> > Status in Zun:<br/> > Bug description:<br/> > I have this error when I exec intercatve container<br/> > # openstack appcontainer exec --interactive container /bin/sh<br/> > HTTPConnectionP<wbr/>ool(host=<wbr/>'130.127.<wbr/>133.20'<wbr/>, port=2375): Max retries<br/> > exceeded with url:<br/> > /v1.35/<wbr/>exec/808ebed184<wbr/>1484c028d0f0684<wbr/>f7637fe28849592<wbr/>125607c55173155<wbr/>ca0f47668/<wbr/>start<br/> > (Caused by NewConnectionEr<wbr/>ror('<urllib3.<wbr/>connection.<wbr/>HTTPConnection<br/> > object at 0x7f44ae9ef550>: Failed to establish a new connection:<br/> > [Errno 111] Connection refused',))<br/> > /etc/systemd/<wbr/>system/<wbr/>docker.<wbr/>service.<wbr/>d/docker.<wbr/>conf #my compute node<br/> > [Service]<br/> > ExecStart=<br/> > ExecStart=<wbr/>/usr/bin/<wbr/>dockerd --group zun -H tcp://compute-<wbr/>1:2375 -H<br/> > unix://<wbr/>/var/run/<wbr/>docker.<wbr/>sock --cluster-store etcd://<wbr/>controller:<wbr/>2379<br/> > To manage notifications about this bug go to:<br/> > <a rel="nofollow" href="https://bugs.launchpad.net/zun/+bug/1830450/+subscriptions">https:/<wbr/>/bugs.launchpad<wbr/>.net/zun/<wbr/>+bug/1830450/<wbr/>+subscriptions</a><br/> > the problem was resolved on the rocky release but I have an other problem: > # openstack appcontainer exec --interactive container /bin/sh > WARNING: Failed to import plugin clustering. > connected to container, press Enter to continue > type ~. to disconnect > Internal Server Error (HTTP 500) (Request-ID: > req-bba1d6e2-d0fe-4915-8522-5d0f9f12d634) Could you add a '--debug' option and show me the output? In addition, do you find some error messages in the Zun logs (i.e. zun-api, zun-compute, zun-wsproxy). # openstack --debug appcontainer exec --interactive container /bin/sh > __________________________________________________________________________________ > and also i have a odoo container running but when I am trying to acces > to it at 10.245.0.11:8069 or 10.245.0.11:8078 via the browser there is > no response > +--------------------------------------+-----------+--------+---------+--------------------+-------------+--------------+ > | uuid | name | image | status | > task_state | addresses | ports | > +--------------------------------------+-----------+--------+---------+--------------------+-------------+--------------+ > | fd4d5923-91f6-4fdd-aa29-5ac5d3813286 | odoo | odoo | Running | > None | 10.254.0.11 | [8069, 8071] | > | 6028d286-9caa-46c9-937e-a37f5925eabf | container | cirros | Created | > container_starting | 10.10.0.20 | [] | > +--------------------------------------+-----------+--------+---------+--------------------+-------------+--------------+ It might relate to security groups. By default, the 'odoo' is using the 'default' security group, so you want to make sure if those ports are open in the security group. Let me know if it doesn't help. > You received this bug notification because you are subscribed to Zun. > Matching subscriptions: zun > https://bugs.launchpad.net/bugs/1830450 > Title: > openstack appcontainer exec --interactive > Status in Zun: > Bug description: > I have this error when I exec intercatve container > # openstack appcontainer exec --interactive container /bin/sh > HTTPConnectionPool(host='130.127.133.20', port=2375): Max retries > exceeded with url: > /v1.35/exec/808ebed1841484c028d0f0684f7637fe28849592125607c55173155ca0f47668/start > (Caused by NewConnectionError('<urllib3.connection.HTTPConnection > object at 0x7f44ae9ef550>: Failed to establish a new connection: > [Errno 111] Connection refused',)) > /etc/systemd/system/docker.service.d/docker.conf #my compute node > [Service] > ExecStart= > ExecStart=/usr/bin/dockerd --group zun -H tcp://compute-1:2375 -H > unix:///var/run/docker.sock --cluster-store etcd://controller:2379 > To manage notifications about this bug go to: > https://bugs.launchpad.net/zun/+bug/1830450/+subscriptions <div class="comment-text editable-message-text" itemprop="commentText"><p>Hi @hongbin</p> <p># openstack --debug appcontainer exec --interactive 11ba273f-<wbr/>e63a-49e2-<wbr/>a6d4-3e2d688d10<wbr/>93 /bin/sh<br/> WARNING: Failed to import plugin clustering.<br/> START with options: --debug appcontainer exec --interactive 11ba273f-<wbr/>e63a-49e2-<wbr/>a6d4-3e2d688d10<wbr/>93 /bin/sh<br/> options: Namespace(<wbr/>access_<wbr/>key='', access_<wbr/>secret=<wbr/>'***', access_token='***', access_<wbr/>token_endpoint=<wbr/>'', access_<wbr/>token_type=<wbr/>'', aodh_endpoint='', application_<wbr/>credential_<wbr/>id='', application_<wbr/>credential_<wbr/>name=''<wbr/>, application_<wbr/>credential_<wbr/>secret=<wbr/>'***', auth_type=<wbr/>'password'<wbr/>, auth_url='<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, cacert=None, cert='', client_id='', client_<wbr/>secret=<wbr/>'***', cloud='', code='', consumer_key='', consumer_<wbr/>secret=<wbr/>'***', debug=True, default_<wbr/>domain=<wbr/>'default'<wbr/>, default_<wbr/>domain_<wbr/>id='', default_<wbr/>domain_<wbr/>name=''<wbr/>, deferred_<wbr/>help=False, discovery_<wbr/>endpoint=<wbr/>'', domain_id='', domain_name='', endpoint='', identity_<wbr/>provider=<wbr/>'', identity_<wbr/>provider_<wbr/>url='', insecure=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_<wbr/>api_version=<wbr/>'2', os_beta_<wbr/>command=<wbr/>False, os_compute_<wbr/>api_version=<wbr/>'', os_container_<wbr/>api_version=<wbr/>'1.20', os_container_<wbr/>infra_api_<wbr/>version=<wbr/>'1', os_data_<wbr/>processing_<wbr/>api_version=<wbr/>'1.1', os_data_<wbr/>processing_<wbr/>url='', os_database_<wbr/>api_version=<wbr/>'1', os_dns_<wbr/>api_version=<wbr/>'2', os_identity_<wbr/>api_version=<wbr/>'3', os_image_<wbr/>api_version=<wbr/>'2', os_key_<wbr/>manager_<wbr/>api_version=<wbr/>'1', os_loadbalancer<wbr/>_api_version=<wbr/>'2.0', os_metrics_<wbr/>api_version=<wbr/>'1', os_network_<wbr/>api_version=<wbr/>'', os_object_<wbr/>api_version=<wbr/>'', os_orchestratio<wbr/>n_api_version=<wbr/>'1', os_project_id=None, os_project_<wbr/>name=None, os_queues_<wbr/>api_version=<wbr/>'2', os_volume_<wbr/>api_version=<wbr/>'', os_workflow_<wbr/>api_version=<wbr/>'2', passcode='', password='***', profile='', project_<wbr/>domain_<wbr/>id='', project_<wbr/>domain_<wbr/>name='default'<wbr/>, project_id='', project_<wbr/>name='admin'<wbr/>, protocol='', redirect_uri='', region_name='', remote_<wbr/>project_<wbr/>domain_<wbr/>id='', remote_<wbr/>project_<wbr/>domain_<wbr/>name=''<wbr/>, remote_<wbr/>project_<wbr/>id='', remote_<wbr/>project_<wbr/>name=''<wbr/>, roles='', service_<wbr/>provider=<wbr/>'', service_<wbr/>provider_<wbr/>endpoint=<wbr/>'', service_<wbr/>provider_<wbr/>entity_<wbr/>id='', system_scope='', timing=False, token='***', trust_id='', url='', user='', user_domain_id='', user_domain_<wbr/>name='default'<wbr/>, user_id='', username=<wbr/>'adminapi'<wbr/>, verbose_level=3, verify=None)<br/> Auth plugin password selected<br/> auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'key': None, 'database_<wbr/>api_version'<wbr/>: '1', 'metrics_<wbr/>api_version'<wbr/>: '1', 'data_processin<wbr/>g_api_version'<wbr/>: '1.1', 'auth_url': '<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, u'network_<wbr/>api_version'<wbr/>: u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_<wbr/>api_version'<wbr/>: '2', 'verify': True, 'dns_api_version': '2', u'object_<wbr/>store_api_<wbr/>version'<wbr/>: u'1', 'username': 'adminapi', 'container_<wbr/>infra_api_<wbr/>version'<wbr/>: '1', 'loadbalancer_<wbr/>api_version'<wbr/>: '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_<wbr/>introspection_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'api_timeout': None, 'image_<wbr/>api_version'<wbr/>: '2', 'auth': {'user_<wbr/>domain_<wbr/>name': 'default', 'project_name': 'admin', 'project_<wbr/>domain_<wbr/>name': 'default'}, 'default_domain': 'default', 'container_<wbr/>api_version'<wbr/>: '1.20', u'image_<wbr/>api_use_<wbr/>tasks': False, u'floating_<wbr/>ip_source'<wbr/>: u'neutron', 'orchestration_<wbr/>api_version'<wbr/>: '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_<wbr/>api_vers.<wbr/>..</p></div> <textarea style="width: 100%" rows="10">Hi @hongbin # openstack --debug appcontainer exec --interactive 11ba273f-e63a-49e2-a6d4-3e2d688d1093 /bin/sh WARNING: Failed to import plugin clustering. START with options: --debug appcontainer exec --interactive 11ba273f-e63a-49e2-a6d4-3e2d688d1093 /bin/sh options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', aodh_endpoint='', application_credential_id='', application_credential_name='', application_credential_secret='***', auth_type='password', auth_url='http://controller:5000/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_api_version='2', os_beta_command=False, os_compute_api_version='', os_container_api_version='1.20', os_container_infra_api_version='1', os_data_processing_api_version='1.1', os_data_processing_url='', os_database_api_version='1', os_dns_api_version='2', os_identity_api_version='3', os_image_api_version='2', os_key_manager_api_version='1', os_loadbalancer_api_version='2.0', os_metrics_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_queues_api_version='2', os_volume_api_version='', os_workflow_api_version='2', passcode='', password='***', profile='', project_domain_id='', project_domain_name='default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='', remote_project_domain_id='', remote_project_domain_name='', remote_project_id='', remote_project_name='', roles='', service_provider='', service_provider_endpoint='', service_provider_entity_id='', system_scope='', timing=False, token='***', trust_id='', url='', user='', user_domain_id='', user_domain_name='default', user_id='', username='adminapi', verbose_level=3, verify=None) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} defaults: {u'auth_type': 'password', u'status': u'active', u'image_status_code_retries': 5, u'baremetal_introspection_status_code_retries': 5, 'api_timeout': None, 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, u'interface': u'public', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'baremetal_status_code_retries': 5, 'verify': True, 'cert': None, u'secgroup_source': u'neutron', u'object_store_api_version': u'1', u'disable_vendor_agent': {}} cloud cfg: {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'orchestration_api_version': '1', 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'container_api_version': '1.20', u'interface': 'public', u'disable_vendor_agent': {}} compute API version 2.1, cmd group openstack.compute.v2 network API version 2, cmd group openstack.network.v2 image API version 2, cmd group openstack.image.v2 volume API version 2, cmd group openstack.volume.v2 identity API version 3, cmd group openstack.identity.v3 object_store API version 1, cmd group openstack.object_store.v1 container API version 1.20, cmd group openstack.container.v1 messaging API version 2, cmd group openstack.messaging.v2 database API version 1, cmd group openstack.database.v1 data_processing API version 1.1, cmd group openstack.data_processing.v1 load_balancer API version 2.0, cmd group openstack.load_balancer.v2 neutronclient API version 2, cmd group openstack.neutronclient.v2 workflow_engine API version 2, cmd group openstack.workflow_engine.v2 container_infra API version 1, cmd group openstack.container_infra.v1 orchestration API version 1, cmd group openstack.orchestration.v1 dns API version 2, cmd group openstack.dns.v2 key_manager API version 1, cmd group openstack.key_manager.v1 metric API version 1, cmd group openstack.metric.v1 alarming API version 2, cmd group openstack.alarming.v2 Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} command: appcontainer exec -> zunclient.osc.v1.containers.ExecContainer (auth=True) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', 'additional_user_agent': [('osc-lib', '1.12.1')], u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} Using auth plugin: password Using parameters {'username': 'adminapi', 'project_name': 'admin', 'user_domain_name': 'default', 'auth_url': 'http://controller:5000/v3', 'password': '***', 'project_domain_name': 'default'} Get auth_ref REQ: curl -g -i -X GET http://controller:5000/v3 -H "Accept: application/json" -H "User-Agent: openstacksdk/0.28.0 keystoneauth1/3.14.0 python-requests/2.22.0 CPython/2.7.15rc1" Starting new HTTP connection (1): controller:5000 http://controller:5000 "GET /v3 HTTP/1.1" 200 250 RESP: [200] Connection: Keep-Alive Content-Length: 250 Content-Type: application/json Date: Mon, 27 May 2019 15:25:21 GMT Keep-Alive: timeout=5, max=100 Server: Apache/2.4.29 (Ubuntu) Vary: X-Auth-Token x-openstack-request-id: req-b9a634ed-3cd0-4f49-b87b-8d9ce694372b RESP BODY: {"version": {"status": "stable", "updated": "2018-10-15T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.11", "links": [{"href": "http://controller:5000/v3/", "rel": "self"}]}} GET call to http://controller:5000/v3 used request id req-b9a634ed-3cd0-4f49-b87b-8d9ce694372b Making authentication request to http://controller:5000/v3/auth/tokens http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": "2019-05-27T19:25:22.000000Z", "project": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a128816d407542ee836bd30d8c36484d"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": [{"url": "http://controller:8774/v2.1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": "http://controller:8774/v2.1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3b4eef8ae0d64c1396386a5557d50560"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": [{"url": "http://controller:9001/v2", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": "http://controller:9001/v2", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": [{"url": "http://controller:9292", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": [{"url": "http://controller:5000/v3", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": "http://controller:5000/v3", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": "keystone"}, {"endpoints": [{"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3827d192568f4b1e8b1901bdedbe1137"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "916be6147a2d4df293ebf375abd6fed2"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "56aebd20ee19475cae24f2e558de4b2d"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": [{"url": "http://controller:9511/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": "http://controller:9511/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", "name": "magnum"}, {"endpoints": [{"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "52504619399d467ba64a1f232129768c"}, {"url": "http://controller:8080/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": [{"url": "http://controller:8041", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": [{"url": "http://controller:8778", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": "ceilometer"}, {"endpoints": [{"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0145b180de764baabb3f8fc15e5e0883"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b6006479263b4faf81963f21f523e11d"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": [{"url": "http://controller:8000/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "7b4e560125674762b7d516605986f5aa"}, {"url": "http://controller:8000/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "04d221684116407981f9d00d177b077a"}, {"url": "http://controller:9517/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "848b8262069a4c4c91716c52644783d8"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "password_expires_at": null, "name": "adminapi", "id": "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": ["ihZUPQm7QyuIbgTMqY3oGg"], "issued_at": "2019-05-27T15:25:22.000000Z"}} run(Namespace(command=[u'/bin/sh'], container=u'11ba273f-e63a-49e2-a6d4-3e2d688d1093', interactive=True)) take_action(Namespace(command=[u'/bin/sh'], container=u'11ba273f-e63a-49e2-a6d4-3e2d688d1093', interactive=True)) Instantiating zun client: <class 'zunclient.v1.client.Client'> Making authentication request to http://controller:5000/v3/auth/tokens http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": "2019-05-27T19:25:22.000000Z", "project": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a128816d407542ee836bd30d8c36484d"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": [{"url": "http://controller:8774/v2.1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": "http://controller:8774/v2.1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3b4eef8ae0d64c1396386a5557d50560"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": [{"url": "http://controller:9001/v2", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": "http://controller:9001/v2", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": [{"url": "http://controller:9292", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": [{"url": "http://controller:5000/v3", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": "http://controller:5000/v3", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": "keystone"}, {"endpoints": [{"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3827d192568f4b1e8b1901bdedbe1137"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "916be6147a2d4df293ebf375abd6fed2"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "56aebd20ee19475cae24f2e558de4b2d"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": [{"url": "http://controller:9511/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": "http://controller:9511/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", "name": "magnum"}, {"endpoints": [{"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "52504619399d467ba64a1f232129768c"}, {"url": "http://controller:8080/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": [{"url": "http://controller:8041", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": [{"url": "http://controller:8778", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": "ceilometer"}, {"endpoints": [{"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0145b180de764baabb3f8fc15e5e0883"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b6006479263b4faf81963f21f523e11d"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": [{"url": "http://controller:8000/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "7b4e560125674762b7d516605986f5aa"}, {"url": "http://controller:8000/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "04d221684116407981f9d00d177b077a"}, {"url": "http://controller:9517/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "848b8262069a4c4c91716c52644783d8"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "password_expires_at": null, "name": "adminapi", "id": "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": ["l2mCXjshRJOm1mmivHBIow"], "issued_at": "2019-05-27T15:25:22.000000Z"}} REQ: curl -g -i -X POST http://controller:9517/v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: application/json" -H "OpenStack-API-Version: container 1.20" -H "User-Agent: None" -H "X-Auth-Token: {SHA256}1cb2e0040ecb422e8b0840d348f06d63d70df8bb1db44491275c68097843401c" Starting new HTTP connection (1): controller:9517 http://controller:9517 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" 200 310 RESP: [200] Connection: keep-alive Content-Length: 310 Content-Type: application/json Date: Mon, 27 May 2019 15:25:23 GMT Openstack-Api-Maximum-Version: container 1.22 Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: req-47d24758-1fbf-4bc5-bd26-4da2ca5c1f20 RESP BODY: {"output": null, "exec_id": "66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160", "proxy_url": "ws://controller:6784/?token=f065e6c5-c1d4-4267-9326-01a23dc24f65&uuid=11ba273f-e63a-49e2-a6d4-3e2d688d1093&exec_id=66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160", "exit_code": null} POST call to container for http://controller:9517/v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True used request id req-47d24758-1fbf-4bc5-bd26-4da2ca5c1f20 connecting to: ws://controller:6784/?token=f065e6c5-c1d4-4267-9326-01a23dc24f65&uuid=11ba273f-e63a-49e2-a6d4-3e2d688d1093&exec_id=66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160 connected to 11ba273f-e63a-49e2-a6d4-3e2d688d1093, press Enter to continue type ~. to disconnect REQ: curl -g -i -X POST http://controller:9517/v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute_resize?h=36&exec_id=66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160&w=142 -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: application/json" -H "OpenStack-API-Version: container 1.20" -H "User-Agent: None" -H "X-Auth-Token: {SHA256}1cb2e0040ecb422e8b0840d348f06d63d70df8bb1db44491275c68097843401c" http://controller:9517 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute_resize?h=36&exec_id=66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160&w=142 HTTP/1.1" 500 261 RESP: [500] Connection: keep-alive Content-Length: 261 Content-Type: application/json Date: Mon, 27 May 2019 15:25:33 GMT Openstack-Api-Maximum-Version: container 1.22 Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: req-0b7cc960-50a8-4022-a89a-19331907277e RESP BODY: {"errors": [{"status": 500, "code": "container.", "links": [], "title": "Internal Server Error", "detail": "Your request could not be handled because of a problem in the server. Error Correlation id is: e44458e8-21e3-4f9e-81fe-f581b57cd09b", "request_id": ""}]} POST call to container for http://controller:9517/v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute_resize?h=36&exec_id=66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160&w=142 used request id req-0b7cc960-50a8-4022-a89a-19331907277e Internal Server Error (HTTP 500) (Request-ID: req-0b7cc960-50a8-4022-a89a-19331907277e) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", line 537, in take_action websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 379, in do_exec wscls.handle_resize() File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 196, in handle_resize self.tty_resize(height=rows, width=cols) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 294, in tty_resize self.cs.containers.execute_resize(self.id, self.exec_id, width, height) File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 165, in execute_resize qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 128, in _action **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 367, in json_request resp = self._http_request(url, method, **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 350, in _http_request error_json.get('debuginfo'), method, url) InternalServerError: Internal Server Error (HTTP 500) (Request-ID: req-0b7cc960-50a8-4022-a89a-19331907277e) clean_up ExecContainer: Internal Server Error (HTTP 500) (Request-ID: req-0b7cc960-50a8-4022-a89a-19331907277e) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 136, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 281, in run result = self.run_subcommand(remainder) File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 176, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", line 537, in take_action websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 379, in do_exec wscls.handle_resize() File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 196, in handle_resize self.tty_resize(height=rows, width=cols) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 294, in tty_resize self.cs.containers.execute_resize(self.id, self.exec_id, width, height) File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 165, in execute_resize qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 128, in _action **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 367, in json_request resp = self._http_request(url, method, **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 350, in _http_request error_json.get('debuginfo'), method, url) InternalServerError: Internal Server Error (HTTP 500) (Request-ID: req-0b7cc960-50a8-4022-a89a-19331907277e) END return value: 1 # cat zun-api.log -- Logs begin at Sun 2019-05-26 18:55:41 EDT, end at Mon 2019-05-27 11:31:27 EDT. -- May 26 19:16:02 controller.zunrocky.labs1-pg0.clemson.cloudlab.us systemd[1]: Started OpenStack Container Service API. May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.089 26254 INFO zun.api.app [-] Full WSGI config used: /etc/zun/api-paste.ini May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.117 26254 WARNING keystonemiddleware._common.config [-] The option "__file__" in conf is not known to auth_token May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.117 26254 WARNING keystonemiddleware._common.config [-] The option "here" in conf is not known to auth_token May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.125 26254 INFO oslo.service.wsgi [-] zun_api listening on 192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.125 26254 INFO oslo_service.service [-] Starting 56 workers May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.135 26263 INFO eventlet.wsgi.server [-] (26263) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.135 26262 INFO eventlet.wsgi.server [-] (26262) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.137 26264 INFO eventlet.wsgi.server [-] (26264) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.140 26265 INFO eventlet.wsgi.server [-] (26265) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.141 26266 INFO eventlet.wsgi.server [-] (26266) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.142 26267 INFO eventlet.wsgi.server [-] (26267) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.145 26268 INFO eventlet.wsgi.server [-] (26268) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.146 26269 INFO eventlet.wsgi.server [-] (26269) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.148 26270 INFO eventlet.wsgi.server [-] (26270) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.150 26271 INFO eventlet.wsgi.server [-] (26271) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.151 26272 INFO eventlet.wsgi.server [-] (26272) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.154 26274 INFO eventlet.wsgi.server [-] (26274) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.153 26273 INFO eventlet.wsgi.server [-] (26273) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.157 26275 INFO eventlet.wsgi.server [-] (26275) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.159 26276 INFO eventlet.wsgi.server [-] (26276) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.160 26277 INFO eventlet.wsgi.server [-] (26277) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.162 26280 INFO eventlet.wsgi.server [-] (26280) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.163 26278 INFO eventlet.wsgi.server [-] (26278) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.163 26279 INFO eventlet.wsgi.server [-] (26279) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.168 26281 INFO eventlet.wsgi.server [-] (26281) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.169 26283 INFO eventlet.wsgi.server [-] (26283) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.169 26282 INFO eventlet.wsgi.server [-] (26282) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.171 26285 INFO eventlet.wsgi.server [-] (26285) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.173 26284 INFO eventlet.wsgi.server [-] (26284) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.176 26286 INFO eventlet.wsgi.server [-] (26286) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.178 26287 INFO eventlet.wsgi.server [-] (26287) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.179 26288 INFO eventlet.wsgi.server [-] (26288) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.181 26289 INFO eventlet.wsgi.server [-] (26289) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.184 26290 INFO eventlet.wsgi.server [-] (26290) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.185 26291 INFO eventlet.wsgi.server [-] (26291) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.187 26292 INFO eventlet.wsgi.server [-] (26292) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.189 26293 INFO eventlet.wsgi.server [-] (26293) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.192 26294 INFO eventlet.wsgi.server [-] (26294) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.193 26295 INFO eventlet.wsgi.server [-] (26295) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.195 26296 INFO eventlet.wsgi.server [-] (26296) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.196 26297 INFO eventlet.wsgi.server [-] (26297) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.198 26298 INFO eventlet.wsgi.server [-] (26298) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.200 26299 INFO eventlet.wsgi.server [-] (26299) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.202 26300 INFO eventlet.wsgi.server [-] (26300) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.205 26301 INFO eventlet.wsgi.server [-] (26301) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.206 26302 INFO eventlet.wsgi.server [-] (26302) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.209 26303 INFO eventlet.wsgi.server [-] (26303) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.211 26304 INFO eventlet.wsgi.server [-] (26304) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.212 26305 INFO eventlet.wsgi.server [-] (26305) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.213 26306 INFO eventlet.wsgi.server [-] (26306) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.215 26308 INFO eventlet.wsgi.server [-] (26308) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.216 26307 INFO eventlet.wsgi.server [-] (26307) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.219 26309 INFO eventlet.wsgi.server [-] (26309) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.221 26310 INFO eventlet.wsgi.server [-] (26310) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.223 26311 INFO eventlet.wsgi.server [-] (26311) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.225 26312 INFO eventlet.wsgi.server [-] (26312) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.226 26314 INFO eventlet.wsgi.server [-] (26314) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.226 26313 INFO eventlet.wsgi.server [-] (26313) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.231 26315 INFO eventlet.wsgi.server [-] (26315) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.232 26316 INFO eventlet.wsgi.server [-] (26316) wsgi starting up on http://192.168.0.1:9517 May 26 19:16:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 19:16:04.234 26317 INFO eventlet.wsgi.server [-] (26317) wsgi starting up on http://192.168.0.1:9517 May 26 20:04:42 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 20:04:42.251 26314 INFO eventlet.wsgi.server [req-b27ea63f-3f12-4b80-b48a-ce2b79a3c0cc - - - - -] 192.168.0.1 "GET / HTTP/1.1" status: 200 len: 641 time: 0.0162020 May 26 20:04:43 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 20:04:43.764 26314 INFO eventlet.wsgi.server [req-31095b98-278e-4a18-b93a-518e9c938990 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 391 time: 0.8385611 May 26 20:05:24 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 20:05:24.869 26285 INFO eventlet.wsgi.server [req-2175eeee-194a-49e6-9a1a-efbc9e93e196 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 391 time: 0.2329068 May 26 20:07:02 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 20:07:02.730 26285 INFO eventlet.wsgi.server [req-2a1facfd-22d5-4503-987b-35579057df8c ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 391 time: 0.0172601 May 26 20:08:34 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 20:08:34.515 26271 INFO eventlet.wsgi.server [req-c14789b3-3bf3-4e64-8b07-e792d4690f6d - - - - -] 192.168.0.1 "GET / HTTP/1.1" status: 200 len: 641 time: 0.0152540 May 26 20:08:35 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 20:08:35.692 26308 INFO eventlet.wsgi.server [req-13531b89-8bff-4c46-a4a2-4923363acfbe f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 391 time: 0.7395310 May 26 20:13:42 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-26 20:13:42.441 26313 INFO eventlet.wsgi.server [req-6af196ff-64f5-40cd-99fa-523649b7ecc2 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 391 time: 0.2603528 May 27 09:58:03 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 09:58:03.340 26274 INFO eventlet.wsgi.server [req-3787e4a4-5e75-40a7-81ad-2a43a90e44d9 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/services HTTP/1.1" status: 200 len: 714 time: 0.8898270 May 27 10:08:40 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:08:40.493 26310 INFO eventlet.wsgi.server [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers?run=true HTTP/1.1" status: 202 len: 1350 time: 1.8265741 May 27 10:08:49 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:08:49.890 26275 INFO eventlet.wsgi.server [req-50455539-9bdd-41b6-bdbe-e0eaace4fbd7 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1355 time: 0.7699239 May 27 10:09:07 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:07.130 26298 INFO eventlet.wsgi.server [req-adf6f933-0428-40e7-ae58-93eb91c79044 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" status: 200 len: 670 time: 0.8317029 May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception [req-0c845c78-e185-4643-93ca-f25981c8304f f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 5f1ad226-7981-4eeb-8d46-8661c1b8e0cd:Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: : DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception Traceback (most recent call last): May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/exception.py", line 110, in wrapped May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return func(*args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/validation/__init__.py", line 55, in wrapper May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return func(*args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/controllers/v1/containers.py", line 925, in execute_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception kwargs.get('w', None)) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception result = f(*args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/api.py", line 145, in container_exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return self.rpcapi.container_exec_resize(context, container, *args) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception result = f(*args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 36, in wrap May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return func(self, context, container, *args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 115, in container_exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception exec_id=exec_id, height=height, width=width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/rpc_service.py", line 100, in _call May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return cctxt.call(self._context, method, *args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception retry=self.retry) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception retry=retry) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception call_monitor_timeout, retry=retry) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception raise result May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception Traceback (most recent call last): May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception res = self.dispatcher.dispatch(message) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception result = func(ctxt, **new_args) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return function(self, context, *args, **kwargs) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception return self.driver.execute_resize(exec_id, height, width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception raise May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception self.gen.throw(type, value, traceback) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception **client_kwargs May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception docker.exec_resize(exec_id, height=height, width=width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception exec_id, height=height, width=width) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception self._raise_for_status(res) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception raise create_api_error_from_http_exception(e) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception raise cls(e, response=response, explanation=explanation) May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.211 26298 ERROR zun.common.exception May 27 10:09:17 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:17.251 26298 INFO eventlet.wsgi.server [req-0c845c78-e185-4643-93ca-f25981c8304f f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/container/execute_resize?h=38&exec_id=03f04710c6e03154e66aba390979e269a72defe619c67e7318839f6705e8de4e&w=142 HTTP/1.1" status: 500 len: 640 time: 10.0950351 May 27 10:09:36 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:36.047 26289 INFO eventlet.wsgi.server [req-aee9e3c2-4fbe-4e3c-8025-c5eec9598d2d ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1339 time: 0.7936649 May 27 10:09:40 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:40.668 26283 INFO eventlet.wsgi.server [req-7b38ebc7-9302-4b97-ad65-62f614d96d98 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/02d577ce-d97b-4f92-be7d-59855aad7dc2 HTTP/1.1" status: 200 len: 1530 time: 0.3532422 May 27 10:09:41 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:09:41.042 26289 INFO eventlet.wsgi.server [req-6826af10-b408-4632-ab8b-511a252c1f4d ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/02d577ce-d97b-4f92-be7d-59855aad7dc2/logs?stderr=True&stdout=True HTTP/1.1" status: 200 len: 360 time: 0.0803311 May 27 10:11:05 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:11:05.633 26301 INFO eventlet.wsgi.server [req-c7726c5a-efd4-45b5-95a7-f30d049c257d ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1345 time: 0.2574439 May 27 10:11:10 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:11:10.790 26302 INFO eventlet.wsgi.server [req-b8ff1769-8ae2-4399-bd99-c42df8db4c7d ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/02d577ce-d97b-4f92-be7d-59855aad7dc2 HTTP/1.1" status: 200 len: 1526 time: 0.3682961 May 27 10:11:11 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:11:11.306 26310 INFO eventlet.wsgi.server [req-1c9dcf18-73a7-42c5-930d-3e7627160fbc ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.0192831 May 27 10:11:58 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:11:58.079 26278 INFO eventlet.wsgi.server [req-cfd889bd-1794-41ab-a43a-92e181a578f7 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1341 time: 0.7486050 May 27 10:12:35 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:12:35.559 26305 INFO eventlet.wsgi.server [req-e7cd7ba7-50aa-4c02-8a48-2c77d9f05d88 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1341 time: 0.2424490 May 27 10:12:39 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:12:39.981 26272 INFO eventlet.wsgi.server [req-52e0140a-cbca-4b61-9e2b-c88081769833 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1341 time: 0.2404280 May 27 10:12:41 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:12:41.898 26307 INFO eventlet.wsgi.server [req-bb9a21e3-cc23-40bf-9a01-1aa747b2e02e ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/02d577ce-d97b-4f92-be7d-59855aad7dc2 HTTP/1.1" status: 200 len: 1526 time: 0.3423009 May 27 10:12:42 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:12:42.603 26266 INFO eventlet.wsgi.server [req-9aeeb0b7-d2e1-4f13-8dd8-6ade361c4507 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.2370839 May 27 10:13:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:04.125 26293 INFO eventlet.wsgi.server [req-5f7e2abb-0835-4b7a-8499-1c8102cf5270 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" status: 200 len: 670 time: 0.8194079 May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception [req-af3460e5-e5e0-4ec4-90bb-bd709ea96a38 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] d4a4cfea-1544-4abc-bf28-576528d12ec3:Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: : DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception Traceback (most recent call last): May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/exception.py", line 110, in wrapped May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return func(*args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/validation/__init__.py", line 55, in wrapper May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return func(*args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/controllers/v1/containers.py", line 925, in execute_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception kwargs.get('w', None)) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception result = f(*args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/api.py", line 145, in container_exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return self.rpcapi.container_exec_resize(context, container, *args) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception result = f(*args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 36, in wrap May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return func(self, context, container, *args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 115, in container_exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception exec_id=exec_id, height=height, width=width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/rpc_service.py", line 100, in _call May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return cctxt.call(self._context, method, *args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception retry=self.retry) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception retry=retry) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception call_monitor_timeout, retry=retry) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception raise result May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception Traceback (most recent call last): May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception res = self.dispatcher.dispatch(message) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception result = func(ctxt, **new_args) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return function(self, context, *args, **kwargs) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception return self.driver.execute_resize(exec_id, height, width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception raise May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception self.gen.throw(type, value, traceback) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception **client_kwargs May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception docker.exec_resize(exec_id, height=height, width=width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception exec_id, height=height, width=width) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception self._raise_for_status(res) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception raise create_api_error_from_http_exception(e) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception raise cls(e, response=response, explanation=explanation) May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.185 26293 ERROR zun.common.exception May 27 10:13:14 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:13:14.190 26293 INFO eventlet.wsgi.server [req-af3460e5-e5e0-4ec4-90bb-bd709ea96a38 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/container/execute_resize?h=38&exec_id=7d984f8c2f51538a929ae1c402e0ab43664e6c7389f2ff9eebf39125f9be6bc5&w=142 HTTP/1.1" status: 500 len: 640 time: 10.0440910 May 27 10:14:13 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:14:13.020 26310 INFO eventlet.wsgi.server [req-ac14b539-9f56-43c6-8a62-f8be99ff2e9a ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/02d577ce-d97b-4f92-be7d-59855aad7dc2/stop?timeout=10 HTTP/1.1" status: 202 len: 368 time: 0.0762060 May 27 10:14:13 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:14:13.270 26283 INFO eventlet.wsgi.server [req-af4f032e-270a-4865-8255-df6d00e4b120 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1357 time: 0.0178621 May 27 10:18:28 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:18:28.162 26290 INFO eventlet.wsgi.server [req-965b875f-4d6d-4965-86bd-56405c5e8051 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1341 time: 0.8173161 May 27 10:33:52 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:33:52.178 26265 INFO eventlet.wsgi.server [req-77e578f9-7ea3-4620-b39b-be1c2830b1c4 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1341 time: 0.9901109 May 27 10:33:56 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:33:56.224 26272 INFO eventlet.wsgi.server [req-9741b4b0-4d92-4608-966e-e369eb089c5e ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/02d577ce-d97b-4f92-be7d-59855aad7dc2 HTTP/1.1" status: 200 len: 1541 time: 0.1217179 May 27 10:33:57 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:33:57.251 26293 INFO eventlet.wsgi.server [req-b4920719-2db8-42b2-8122-6ae4e6cf7190 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.0168509 May 27 10:37:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:37:59.234 26272 INFO eventlet.wsgi.server [req-15284a1c-d74e-4bac-b0bb-672509055fa3 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 1356 time: 0.0191102 May 27 10:38:01 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:38:01.280 26269 INFO eventlet.wsgi.server [req-b38a8013-7fcf-47b2-b0ad-f9ea87a45b26 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.2471571 May 27 10:39:25 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:39:25.281 26272 INFO eventlet.wsgi.server [req-dd889232-4e8b-41a2-a72b-89d3336fc79a ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers?run=true HTTP/1.1" status: 400 len: 605 time: 0.7725589 May 27 10:39:32 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:39:32.915 26304 INFO eventlet.wsgi.server [req-5c62c8e3-f18b-4666-b5db-1102b3a22883 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.2574229 May 27 10:40:27 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:40:27.341 26262 INFO eventlet.wsgi.server [req-668bae22-424f-49a3-9c82-8f59fa9aeb96 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers?run=true HTTP/1.1" status: 400 len: 605 time: 0.3757451 May 27 10:41:55 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:41:55.643 26272 INFO eventlet.wsgi.server [req-db1b8b88-3263-485d-a45f-856c3e2d6f99 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.0162649 May 27 10:42:41 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:42:41.634 26312 INFO eventlet.wsgi.server [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers?run=true HTTP/1.1" status: 202 len: 1371 time: 2.3053391 May 27 10:42:42 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:42:42.094 26281 INFO eventlet.wsgi.server [req-1698b622-e567-497c-b39b-89df282a75f0 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2106 time: 0.2425799 May 27 10:43:22 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:43:22.268 26281 INFO eventlet.wsgi.server [req-bb3993ee-8e64-40aa-bb84-a27124dab2db f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2106 time: 0.4824610 May 27 10:43:45 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:43:45.767 26307 INFO eventlet.wsgi.server [req-bcffa34c-e9cf-480d-9e07-c3cbfc25c778 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2111 time: 0.5042698 May 27 10:57:19 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 10:57:19.112 26271 INFO eventlet.wsgi.server [req-7b044ad3-46aa-4392-9e3c-d20357803c1a f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2970 time: 0.8351700 May 27 11:08:16 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:08:16.437 26269 INFO eventlet.wsgi.server [req-d83d03b2-8a6f-4ba0-aa25-e83655aa7646 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2970 time: 0.0212970 May 27 11:08:19 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:08:19.065 26272 INFO eventlet.wsgi.server [req-c600678c-80e3-40fe-8031-9f7df7daa237 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d HTTP/1.1" status: 200 len: 2191 time: 0.0800900 May 27 11:08:19 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:08:19.448 26298 INFO eventlet.wsgi.server [req-a182f3ba-4509-49b7-acbd-c6d1a664d4d7 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d/logs?stderr=True&stdout=True HTTP/1.1" status: 200 len: 1194 time: 0.0782871 May 27 11:08:19 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:08:19.728 26317 INFO eventlet.wsgi.server [req-7153aae1-3d3d-4c4e-a912-e2950391b7b7 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d/attach HTTP/1.1" status: 200 len: 468 time: 0.3424039 May 27 11:14:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:14:09.403 26288 INFO eventlet.wsgi.server [req-a3da1253-fba7-4881-a02c-b55b607c582a ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2973 time: 0.9606521 May 27 11:14:11 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:14:11.779 26307 INFO eventlet.wsgi.server [req-310ccf91-ae37-45d0-b3b7-30bff51a78e2 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d HTTP/1.1" status: 200 len: 2191 time: 0.0958900 May 27 11:14:12 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:14:12.157 26290 INFO eventlet.wsgi.server [req-4e3ff49f-9ae3-4bd8-8c1e-17d26dced87e ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d/logs?stderr=True&stdout=True HTTP/1.1" status: 200 len: 1194 time: 0.0713680 May 27 11:14:12 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:14:12.168 26272 INFO eventlet.wsgi.server [req-82ee69f0-e8d4-4080-87aa-2194e43ad116 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d/attach HTTP/1.1" status: 200 len: 468 time: 0.0734639 May 27 11:14:19 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:14:19.989 26271 INFO eventlet.wsgi.server [req-c89da345-26b8-4544-875c-95f68d95e84b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2973 time: 0.0209389 May 27 11:14:25 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:14:25.755 26304 INFO eventlet.wsgi.server [req-5ccd31b4-1261-48c6-9b42-47a50acd3bef ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/02d577ce-d97b-4f92-be7d-59855aad7dc2 HTTP/1.1" status: 200 len: 1541 time: 0.1171710 May 27 11:14:26 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:14:26.300 26269 INFO eventlet.wsgi.server [req-ce66c44e-f69e-41df-a078-2eaeb90003af ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.0157292 May 27 11:15:08 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:15:08.391 26290 INFO eventlet.wsgi.server [req-1da5e9b1-a9b0-48b0-9f6c-2c28e2bc8755 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d HTTP/1.1" status: 200 len: 2191 time: 0.0740011 May 27 11:15:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:15:09.359 26268 INFO eventlet.wsgi.server [req-25bccf45-0c48-4b46-b994-e6aa791a52a4 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.2982070 May 27 11:18:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:18:04.812 26295 INFO eventlet.wsgi.server [req-01352da9-354d-459f-95d5-22bd706007ad f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2973 time: 0.7484260 May 27 11:20:54 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:20:54.585 26312 INFO eventlet.wsgi.server [req-297eed31-9e52-440d-914e-6f27d1d6030c f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 2973 time: 0.5214400 May 27 11:22:51 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:22:51.815 26298 INFO eventlet.wsgi.server [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers?run=true HTTP/1.1" status: 202 len: 1350 time: 1.7696979 May 27 11:22:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:22:59.170 26307 INFO eventlet.wsgi.server [req-4ef1ba9b-7fb7-4275-898b-a3ffe803e4cf f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 3938 time: 0.0859580 May 27 11:23:10 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:23:10.605 26308 INFO eventlet.wsgi.server [req-bdfab207-c6f0-4f29-9e4e-083651472693 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" status: 409 len: 643 time: 0.4844480 May 27 11:23:21 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:23:21.244 26302 INFO eventlet.wsgi.server [req-07b2704a-c636-4feb-b3ef-a95e9ed4562a f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 3922 time: 0.5058551 May 27 11:23:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:23:59.495 26281 INFO eventlet.wsgi.server [req-0e8eaee5-dd3e-4d59-a56d-730c0fe5cc9e f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" status: 200 len: 670 time: 0.1766171 May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception [req-24157610-da7b-4c1f-bec0-a9360a7eb4d8 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 330e5f95-d1c0-4ef8-8756-cce209eea5ad:Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: : DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception Traceback (most recent call last): May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/exception.py", line 110, in wrapped May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return func(*args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/validation/__init__.py", line 55, in wrapper May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return func(*args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/controllers/v1/containers.py", line 925, in execute_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception kwargs.get('w', None)) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception result = f(*args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/api.py", line 145, in container_exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return self.rpcapi.container_exec_resize(context, container, *args) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception result = f(*args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 36, in wrap May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return func(self, context, container, *args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 115, in container_exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception exec_id=exec_id, height=height, width=width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/rpc_service.py", line 100, in _call May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return cctxt.call(self._context, method, *args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception retry=self.retry) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception retry=retry) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception call_monitor_timeout, retry=retry) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception raise result May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception Traceback (most recent call last): May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception res = self.dispatcher.dispatch(message) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception result = func(ctxt, **new_args) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return function(self, context, *args, **kwargs) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception return self.driver.execute_resize(exec_id, height, width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception raise May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception self.gen.throw(type, value, traceback) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception **client_kwargs May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception docker.exec_resize(exec_id, height=height, width=width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception exec_id, height=height, width=width) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception self._raise_for_status(res) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception raise create_api_error_from_http_exception(e) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception raise cls(e, response=response, explanation=explanation) May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.557 26281 ERROR zun.common.exception May 27 11:24:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:09.561 26281 INFO eventlet.wsgi.server [req-24157610-da7b-4c1f-bec0-a9360a7eb4d8 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute_resize?h=36&exec_id=dbb6991d4904e948f3e528d5a218638bfcef99dba8bf2ce51b1248a4fefefaa9&w=142 HTTP/1.1" status: 500 len: 640 time: 10.0446739 May 27 11:24:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:24:59.251 26275 INFO eventlet.wsgi.server [req-8bb4b470-b3da-4fc7-b8b6-7e5df1a7c049 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" status: 200 len: 670 time: 0.3144510 May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception [req-21b3e2e6-3d14-47bd-8ee0-3c88782f29fc f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] b0b04ed6-ad54-4e6c-b178-421905b8b53e:Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: : DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception Traceback (most recent call last): May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/exception.py", line 110, in wrapped May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return func(*args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/validation/__init__.py", line 55, in wrapper May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return func(*args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/controllers/v1/containers.py", line 925, in execute_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception kwargs.get('w', None)) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception result = f(*args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/api.py", line 145, in container_exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return self.rpcapi.container_exec_resize(context, container, *args) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception result = f(*args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 36, in wrap May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return func(self, context, container, *args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 115, in container_exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception exec_id=exec_id, height=height, width=width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/rpc_service.py", line 100, in _call May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return cctxt.call(self._context, method, *args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception retry=self.retry) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception retry=retry) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception call_monitor_timeout, retry=retry) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception raise result May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception Traceback (most recent call last): May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception res = self.dispatcher.dispatch(message) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception result = func(ctxt, **new_args) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return function(self, context, *args, **kwargs) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception return self.driver.execute_resize(exec_id, height, width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception raise May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception self.gen.throw(type, value, traceback) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception **client_kwargs May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception exec_id, height=height, width=width) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception self._raise_for_status(res) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception raise create_api_error_from_http_exception(e) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception raise cls(e, response=response, explanation=explanation) May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.307 26275 ERROR zun.common.exception May 27 11:25:09 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:09.313 26275 INFO eventlet.wsgi.server [req-21b3e2e6-3d14-47bd-8ee0-3c88782f29fc f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute_resize?h=36&exec_id=1e59a016bf171f2c9974626bce5ea56c5f5578e8d865200f52510774ab71e1be&w=142 HTTP/1.1" status: 500 len: 640 time: 10.0446372 May 27 11:25:23 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:23.261 26317 INFO eventlet.wsgi.server [req-47d24758-1fbf-4bc5-bd26-4da2ca5c1f20 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" status: 200 len: 670 time: 0.5629599 May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception [req-0b7cc960-50a8-4022-a89a-19331907277e f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] e44458e8-21e3-4f9e-81fe-f581b57cd09b:Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: : DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: Traceback (most recent call last): May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: res = self.dispatcher.dispatch(message) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: result = func(ctxt, **new_args) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return function(self, context, *args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: return self.driver.execute_resize(exec_id, height, width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self.gen.throw(type, value, traceback) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: **client_kwargs May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: exec_id, height=height, width=width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: self._raise_for_status(res) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise create_api_error_from_http_exception(e) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: raise cls(e, response=response, explanation=explanation) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception Traceback (most recent call last): May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/exception.py", line 110, in wrapped May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return func(*args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/validation/__init__.py", line 55, in wrapper May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return func(*args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/api/controllers/v1/containers.py", line 925, in execute_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception kwargs.get('w', None)) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception result = f(*args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/api.py", line 145, in container_exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return self.rpcapi.container_exec_resize(context, container, *args) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 158, in wrapper May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception result = f(*args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 36, in wrap May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return func(self, context, container, *args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/rpcapi.py", line 115, in container_exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception exec_id=exec_id, height=height, width=width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/rpc_service.py", line 100, in _call May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return cctxt.call(self._context, method, *args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception retry=self.retry) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception retry=retry) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception call_monitor_timeout, retry=retry) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception raise result May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception DockerError_Remote: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception Traceback (most recent call last): May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception res = self.dispatcher.dispatch(message) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception result = func(ctxt, **new_args) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return function(self, context, *args, **kwargs) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception return self.driver.execute_resize(exec_id, height, width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception raise May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception self.gen.throw(type, value, traceback) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception **client_kwargs May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception exec_id, height=height, width=width) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception self._raise_for_status(res) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception raise create_api_error_from_http_exception(e) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception raise cls(e, response=response, explanation=explanation) May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.318 26317 ERROR zun.common.exception May 27 11:25:33 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:25:33.323 26317 INFO eventlet.wsgi.server [req-0b7cc960-50a8-4022-a89a-19331907277e f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "POST /v1/containers/11ba273f-e63a-49e2-a6d4-3e2d688d1093/execute_resize?h=36&exec_id=66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160&w=142 HTTP/1.1" status: 500 len: 640 time: 10.0411999 May 27 11:31:20 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:31:20.116 26291 INFO eventlet.wsgi.server [req-64d817b3-902c-459e-8ddc-8a62732310f2 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers HTTP/1.1" status: 200 len: 3922 time: 0.7073030 May 27 11:31:26 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:31:26.117 26304 INFO eventlet.wsgi.server [req-1341ee41-7081-43ed-9a37-bce6689f0a97 ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/containers/d7c8bbaf-0047-4e66-9e28-90b29a7e786d HTTP/1.1" status: 200 len: 2191 time: 0.0811460 May 27 11:31:27 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-api[26254]: 2019-05-27 11:31:27.157 26262 INFO eventlet.wsgi.server [req-7cba2cac-31db-473f-af49-4e30d1ccf46e ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] 192.168.0.1 "GET /v1/availability_zones HTTP/1.1" status: 200 len: 428 time: 0.2774689 # cat zun-wsproxy.log -- Logs begin at Sun 2019-05-26 18:55:41 EDT, end at Mon 2019-05-27 11:35:01 EDT. -- May 26 19:16:07 controller.zunrocky.labs1-pg0.clemson.cloudlab.us systemd[1]: Started OpenStack Container Service Websocket Proxy. May 26 19:16:08 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-26 19:16:08.394 26330 INFO zun.cmd.wsproxy [-] start websocket proxy May 26 19:16:08 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-26 19:16:08.395 26330 INFO zun.websocket.websocketproxy [-] WebSocket server settings: May 26 19:16:08 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-26 19:16:08.395 26330 INFO zun.websocket.websocketproxy [-] - Listen on 192.168.0.1:6784 May 26 19:16:08 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-26 19:16:08.395 26330 INFO zun.websocket.websocketproxy [-] - Flash security policy server May 26 19:16:08 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-26 19:16:08.396 26330 INFO zun.websocket.websocketproxy [-] - SSL/TLS support May 26 19:16:08 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-26 19:16:08.396 26330 INFO zun.websocket.websocketproxy [-] - proxying from 192.168.0.1:6784 to None:None May 27 10:09:07 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:09:07.148 47902 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 10:09:07] 192.168.0.1: Plain non-SSL (ws://) WebSocket connection May 27 10:09:07 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:09:07.150 47902 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 10:09:07] 192.168.0.1: Version hybi-13, base64: 'False' May 27 10:09:07 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:09:07.150 47902 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 10:09:07] 192.168.0.1: Path: '/?token=fb97b23c-31ba-4335-bdb1-098b421cf5c0&uuid=02d577ce-d97b-4f92-be7d-59855aad7dc2&exec_id=03f04710c6e03154e66aba390979e269a72defe619c67e7318839f6705e8de4e' May 27 10:09:07 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:09:07.417 47902 INFO zun.websocket.websocketproxy [req-15d4d06e-2be6-49e5-bc2e-72f14b4341a9 - - - - -] handler exception: HTTPConnectionPool(host='130.127.133.152', port=2375): Max retries exceeded with url: /v1.30/exec/03f04710c6e03154e66aba390979e269a72defe619c67e7318839f6705e8de4e/start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02a3752890>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',)) May 27 10:13:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:13:04.140 48075 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 10:13:04] 192.168.0.1: Plain non-SSL (ws://) WebSocket connection May 27 10:13:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:13:04.141 48075 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 10:13:04] 192.168.0.1: Version hybi-13, base64: 'False' May 27 10:13:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:13:04.141 48075 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 10:13:04] 192.168.0.1: Path: '/?token=5b4d6922-c159-4b93-839e-80a6b1b03b09&uuid=02d577ce-d97b-4f92-be7d-59855aad7dc2&exec_id=7d984f8c2f51538a929ae1c402e0ab43664e6c7389f2ff9eebf39125f9be6bc5' May 27 10:13:04 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:13:04.379 48075 INFO zun.websocket.websocketproxy [req-42e1f263-fc0a-472a-9bd8-98f499addaf3 - - - - -] handler exception: HTTPConnectionPool(host='130.127.133.152', port=2375): Max retries exceeded with url: /v1.30/exec/7d984f8c2f51538a929ae1c402e0ab43664e6c7389f2ff9eebf39125f9be6bc5/start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02a3752950>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',)) May 27 11:23:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:23:59.509 49913 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:23:59] 192.168.0.1: Plain non-SSL (ws://) WebSocket connection May 27 11:23:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:23:59.510 49913 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:23:59] 192.168.0.1: Version hybi-13, base64: 'False' May 27 11:23:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:23:59.510 49913 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:23:59] 192.168.0.1: Path: '/?token=09ff242d-8d8c-47d1-bd4b-dc916523fafb&uuid=11ba273f-e63a-49e2-a6d4-3e2d688d1093&exec_id=dbb6991d4904e948f3e528d5a218638bfcef99dba8bf2ce51b1248a4fefefaa9' May 27 11:23:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:23:59.757 49913 INFO zun.websocket.websocketproxy [req-26ae10a7-adfd-40b7-a57b-94fb8957c25e - - - - -] handler exception: HTTPConnectionPool(host='130.127.133.152', port=2375): Max retries exceeded with url: /v1.30/exec/dbb6991d4904e948f3e528d5a218638bfcef99dba8bf2ce51b1248a4fefefaa9/start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02a3752890>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',)) May 27 11:24:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:24:59.265 49939 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:24:59] 192.168.0.1: Plain non-SSL (ws://) WebSocket connection May 27 11:24:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:24:59.266 49939 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:24:59] 192.168.0.1: Version hybi-13, base64: 'False' May 27 11:24:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:24:59.266 49939 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:24:59] 192.168.0.1: Path: '/?token=efa53c44-e3e1-4b6b-8215-1bc53cbd3747&uuid=11ba273f-e63a-49e2-a6d4-3e2d688d1093&exec_id=1e59a016bf171f2c9974626bce5ea56c5f5578e8d865200f52510774ab71e1be' May 27 11:24:59 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:24:59.523 49939 INFO zun.websocket.websocketproxy [req-90380176-97d0-488b-87cd-8541512568f8 - - - - -] handler exception: HTTPConnectionPool(host='130.127.133.152', port=2375): Max retries exceeded with url: /v1.30/exec/1e59a016bf171f2c9974626bce5ea56c5f5578e8d865200f52510774ab71e1be/start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02a3750bd0>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',)) May 27 11:25:23 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:25:23.274 49959 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:25:23] 192.168.0.1: Plain non-SSL (ws://) WebSocket connection May 27 11:25:23 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:25:23.275 49959 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:25:23] 192.168.0.1: Version hybi-13, base64: 'False' May 27 11:25:23 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:25:23.275 49959 INFO zun.websocket.websocketproxy [-] controller - - [27/May/2019 11:25:23] 192.168.0.1: Path: '/?token=f065e6c5-c1d4-4267-9326-01a23dc24f65&uuid=11ba273f-e63a-49e2-a6d4-3e2d688d1093&exec_id=66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160' May 27 11:25:23 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 11:25:23.516 49959 INFO zun.websocket.websocketproxy [req-ac176378-d53f-447c-93f6-9e8a4ed223c5 - - - - -] handler exception: HTTPConnectionPool(host='130.127.133.152', port=2375): Max retries exceeded with url: /v1.30/exec/66d62708ebaac2d9d8afa5f2075055c60678d7d0f8219f3943d402b2aa374160/start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02a3752bd0>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',)) # cat zun-compute.log -- Logs begin at Sun 2019-05-26 18:55:08 EDT, end at Mon 2019-05-27 11:35:02 EDT. -- May 26 19:33:59 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us systemd[1]: Started OpenStack Container Service Compute Agent. May 26 19:33:59 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:33:59.899 36721 INFO zun.cmd.compute [-] Starting server in PID 36721 May 26 19:33:59 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:33:59.919 36721 INFO zun.container.driver [-] Loading container driver 'docker.driver.DockerDriver' May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:00.114 36721 INFO zun.volume.driver [-] Loading volume driver 'cinder' May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:00.115 36721 INFO zun.image.driver [-] Loading container image driver 'glance' May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:00.116 36721 INFO zun.image.driver [-] Loading container image driver 'docker' May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: /usr/lib/python2.7/dist-packages/sqlalchemy/sql/default_comparator.py:161: SAWarning: The IN-predicate on "container.uuid" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance. May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 'strategies for improved performance.' % expr) May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:00.425 36721 WARNING zun.compute.compute_node_tracker [-] No compute node record for: compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us: ComputeNodeNotFound: Compute node compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us could not be found. May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:00.519 36721 INFO zun.compute.compute_node_tracker [-] Node created for :compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:00.603 36721 INFO oslo.privsep.daemon [-] Running privsep helper: ['sudo', 'zun-rootwrap', '/etc/zun/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/zun/zun.conf', '--privsep_context', 'zun.common.privileged.default', '--privsep_sock_path', '/tmp/tmpBdX3ij/privsep.sock'] May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us sudo[36770]: zun : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/zun-rootwrap /etc/zun/rootwrap.conf privsep-helper --config-file /etc/zun/zun.conf --privsep_context zun.common.privileged.default --privsep_sock_path /tmp/tmpBdX3ij/privsep.sock May 26 19:34:00 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us sudo[36770]: pam_unix(sudo:session): session opened for user root by (uid=0) May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us sudo[36770]: pam_unix(sudo:session): session closed for user root May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.196 36721 INFO oslo.privsep.daemon [-] Spawned new privsep daemon via rootwrap May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.143 36780 INFO oslo.privsep.daemon [-] privsep daemon starting May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.148 36780 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.153 36780 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.153 36780 INFO oslo.privsep.daemon [-] privsep daemon running as pid 36780 May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.453 36780 INFO keyring.backend [-] Loading Windows (alt) May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.460 36780 INFO keyring.backend [-] Loading pyfs May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.461 36780 INFO keyring.backend [-] Loading multi May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.461 36780 INFO keyring.backend [-] Loading Google May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.462 36780 INFO keyring.backend [-] Loading Gnome May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.465 36780 INFO keyring.backend [-] Loading keyczar May 26 19:34:01 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-26 19:34:01.465 36780 INFO keyring.backend [-] Loading file May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.498 36721 WARNING zun.compute.manager [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Ignore the configured default disk size because the driver does not support disk quota. May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.549 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Attempting claim: memory 2048, cpu 1.00 CPU, disk 0 May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.549 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total memory: 257854 MB, used: 0.00 MB May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.549 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] memory limit: 257854.00 MB, free: 257854.00 MB May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.550 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total vcpu: 56 VCPU, used: 0.00 VCPU May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.550 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] vcpu limit: 56.00 VCPU, free: 56.00 VCPU May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.550 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total disk: 12 GB, used: 0.00 GB May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.550 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] disk limit not specified, defaulting to unlimited May 27 10:08:40 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:08:40.551 36721 INFO zun.compute.claims [req-37c126ce-ee61-4ac4-9e04-d5b40f8a4761 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Claim successful May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.187 36721 ERROR zun.compute.manager [req-0c845c78-e185-4643-93ca-f25981c8304f f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Error occurred while calling Docker exec API: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready").: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server [req-0c845c78-e185-4643-93ca-f25981c8304f f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Exception during message handling: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server Traceback (most recent call last): May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server return self.driver.execute_resize(exec_id, height, width) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server raise May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server self.gen.throw(type, value, traceback) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server **client_kwargs May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server docker.exec_resize(exec_id, height=height, width=width) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server exec_id, height=height, width=width) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server self._raise_for_status(res) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:09:17 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:09:17.189 36721 ERROR oslo_messaging.rpc.server May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.178 36721 ERROR zun.compute.manager [req-af3460e5-e5e0-4ec4-90bb-bd709ea96a38 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Error occurred while calling Docker exec API: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready").: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server [req-af3460e5-e5e0-4ec4-90bb-bd709ea96a38 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Exception during message handling: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server Traceback (most recent call last): May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server return self.driver.execute_resize(exec_id, height, width) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server raise May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server self.gen.throw(type, value, traceback) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server **client_kwargs May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server docker.exec_resize(exec_id, height=height, width=width) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server exec_id, height=height, width=width) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server self._raise_for_status(res) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 10:13:14 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:13:14.179 36721 ERROR oslo_messaging.rpc.server May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.640 36721 WARNING zun.compute.manager [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Ignore the configured default disk size because the driver does not support disk quota. May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.717 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Attempting claim: memory 2048, cpu 1.00 CPU, disk 0 May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.718 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total memory: 257854 MB, used: 2048.00 MB May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.719 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] memory limit: 257854.00 MB, free: 255806.00 MB May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.719 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total vcpu: 56 VCPU, used: 1.00 VCPU May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.720 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] vcpu limit: 56.00 VCPU, free: 55.00 VCPU May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.720 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total disk: 12 GB, used: 0.00 GB May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.721 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] disk limit not specified, defaulting to unlimited May 27 10:42:41 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 10:42:41.721 36721 INFO zun.compute.claims [req-f9fb5931-0d59-4c7b-b1fd-c4864588ec3b ba38193f08844cc98c2c870437cc6bd9 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Claim successful May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.819 36721 WARNING zun.compute.manager [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Ignore the configured default disk size because the driver does not support disk quota. May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.876 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Attempting claim: memory 2048, cpu 1.00 CPU, disk 0 May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.877 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total memory: 257854 MB, used: 4096.00 MB May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.878 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] memory limit: 257854.00 MB, free: 253758.00 MB May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.878 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total vcpu: 56 VCPU, used: 2.00 VCPU May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.879 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] vcpu limit: 56.00 VCPU, free: 54.00 VCPU May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.880 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Total disk: 12 GB, used: 0.00 GB May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.880 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] disk limit not specified, defaulting to unlimited May 27 11:22:51 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:22:51.881 36721 INFO zun.compute.claims [req-9f845c8e-456e-4ce9-92cb-162835ab66de f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Claim successful May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.549 36721 ERROR zun.compute.manager [req-24157610-da7b-4c1f-bec0-a9360a7eb4d8 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Error occurred while calling Docker exec API: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready").: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server [req-24157610-da7b-4c1f-bec0-a9360a7eb4d8 f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Exception during message handling: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server Traceback (most recent call last): May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server return self.driver.execute_resize(exec_id, height, width) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server raise May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server self.gen.throw(type, value, traceback) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server **client_kwargs May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server docker.exec_resize(exec_id, height=height, width=width) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server exec_id, height=height, width=width) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server self._raise_for_status(res) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:24:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:24:09.550 36721 ERROR oslo_messaging.rpc.server May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.299 36721 ERROR zun.compute.manager [req-21b3e2e6-3d14-47bd-8ee0-3c88782f29fc f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Error occurred while calling Docker exec API: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready").: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server [req-21b3e2e6-3d14-47bd-8ee0-3c88782f29fc f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Exception during message handling: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server Traceback (most recent call last): May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server return self.driver.execute_resize(exec_id, height, width) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server raise May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server self.gen.throw(type, value, traceback) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server **client_kwargs May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server exec_id, height=height, width=width) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server self._raise_for_status(res) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:09 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:09.300 36721 ERROR oslo_messaging.rpc.server May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.310 36721 ERROR zun.compute.manager [req-0b7cc960-50a8-4022-a89a-19331907277e f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Error occurred while calling Docker exec API: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready").: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server [req-0b7cc960-50a8-4022-a89a-19331907277e f59c50813bb24aad9c7a61a3392a9805 75e35a49fbbe4c049b12ac80a6f21caa 93406f7e7b4a44a3af7b9bfca3f42c66 - -] Exception during message handling: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server Traceback (most recent call last): May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/common/utils.py", line 212, in decorated_function May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/compute/manager.py", line 818, in container_exec_resize May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server return self.driver.execute_resize(exec_id, height, width) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 817, in execute_resize May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server raise May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server self.gen.throw(type, value, traceback) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 49, in docker_client May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 45, in docker_client May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server **client_kwargs May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/driver.py", line 812, in execute_resize May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server docker.exec_resize(exec_id, height=height, width=width) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/zun/container/docker/utils.py", line 98, in exec_resize May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server exec_id, height=height, width=width) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/exec_api.py", line 117, in exec_resize May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server self._raise_for_status(res) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server DockerError: Docker internal error: 500 Server Error: Internal Server Error ("timeout waiting for exec session ready"). May 27 11:25:33 compute-1.zunrocky.labs1-pg0.clemson.cloudlab.us zun-compute[36721]: 2019-05-27 11:25:33.311 36721 ERROR oslo_messaging.rpc.server ______________________________________________________________________ Fod odoo container I use the security group rule as show Egress IPv4 Any Any 0.0.0.0/0 - - Egress IPv6 Any Any ::/0 - - Ingress IPv4 TCP 22 (SSH) 0.0.0.0/0 - - Ingress IPv4 TCP 80 (HTTP) 0.0.0.0/0 - - Ingress IPv4 TCP 443 (HTTPS) 0.0.0.0/0 - - Ingress IPv4 TCP 8069 0.0.0.0/0 - - Ingress IPv4 UDP 161 0.0.0.0/0 - - <div class="comment-text editable-message-text" itemprop="commentText"><p>Hi @hongbin,</p> <p>I can't correct the problem, when I run on Compute Host:# docker exec --interactive zun-ca694bec-<wbr/>2e89-4a3f-<wbr/>b78b-ad21c9fe93<wbr/>53 /bin/bash it work perfectly</p> <p>But on the controller # openstack --debug appcontainer exec --interactive container /bin/sh it give Internal Server Error (HTTP 500)<br/> _______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>____<br/> #openstack --debug appcontainer exec --interactive container /bin/sh<br/> WARNING: Failed to import plugin clustering.<br/> START with options: --debug appcontainer exec --interactive container /bin/sh<br/> options: Namespace(<wbr/>access_<wbr/>key='', access_<wbr/>secret=<wbr/>'***', access_token='***', access_<wbr/>token_endpoint=<wbr/>'', access_<wbr/>token_type=<wbr/>'', aodh_endpoint='', application_<wbr/>credential_<wbr/>id='', application_<wbr/>credential_<wbr/>name=''<wbr/>, application_<wbr/>credential_<wbr/>secret=<wbr/>'***', auth_type=<wbr/>'password'<wbr/>, auth_url='<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, cacert=None, cert='', client_id='', client_<wbr/>secret=<wbr/>'***', cloud='', code='', consumer_key='', consumer_<wbr/>secret=<wbr/>'***', debug=True, default_<wbr/>domain=<wbr/>'default'<wbr/>, default_<wbr/>domain_<wbr/>id='', default_<wbr/>domain_<wbr/>name=''<wbr/>, deferred_<wbr/>help=False, discovery_<wbr/>endpoint=<wbr/>'', domain_id='', domain_name='', endpoint='', identity_<wbr/>provider=<wbr/>'', identity_<wbr/>provider_<wbr/>url='', insecure=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_<wbr/>api_version=<wbr/>'2', os_beta_<wbr/>command=<wbr/>False, os_compute_<wbr/>api_version=<wbr/>'', os_container_<wbr/>api_version=<wbr/>'1.20', os_container_<wbr/>infra_api_<wbr/>version=<wbr/>'1', os_data_<wbr/>processing_<wbr/>api_version=<wbr/>'1.1', os_data_<wbr/>processing_<wbr/>url='', os_database_<wbr/>api_version=<wbr/>'1', os_dns_<wbr/>api_version=<wbr/>'2', os_identity_<wbr/>api_version=<wbr/>'3', os_image_<wbr/>api_version=<wbr/>'2', os_key_<wbr/>manager_<wbr/>api_version=<wbr/>'1', os_loadbalancer<wbr/>_api_version=<wbr/>'2.0', os_metrics_<wbr/>api_version=<wbr/>'1', os_network_<wbr/>api_version=<wbr/>'', os_object_<wbr/>api_version=<wbr/>'', os_orchestratio<wbr/>n_api_version=<wbr/>'1', os_project_id=None, os_project_<wbr/>name=None, os_queues_<wbr/>api_version=<wbr/>'2', os_volume_<wbr/>api_version=<wbr/>'', os_workflow_<wbr/>api_version=<wbr/>'2', passcode='', password='***', profile='', project_<wbr/>domain_<wbr/>id='', project_<wbr/>domain_<wbr/>name='default'<wbr/>, project_id='', project_<wbr/>name='admin'<wbr/>, protocol='', redirect_uri='', region_name='', remote_<wbr/>project_<wbr/>domain_<wbr/>id='', remote_<wbr/>project_<wbr/>domain_<wbr/>name=''<wbr/>, remote_<wbr/>project_<wbr/>id='', remote_<wbr/>project_<wbr/>name=''<wbr/>, roles='', service_<wbr/>provider=<wbr/>'', service_<wbr/>provider_<wbr/>endpoint=<wbr/>'', service_<wbr/>provider_<wbr/>entity_<wbr/>id='', system_scope='', timing=False, token='***', trust_id='', url='', user='', user_domain_id='', user_domain_<wbr/>name='default'<wbr/>, user_id='', username=<wbr/>'adminapi'<wbr/>, verbose_level=3, verify=None)<br/> Auth plugin password selected<br/> auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'key': None, 'database_<wbr/>api_version'<wbr/>: '1', 'metrics_<wbr/>api_version'<wbr/>: '1', 'data_processin<wbr/>g_api_version'<wbr/>: '1.1', 'auth_url': '<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, u'network_<wbr/>api_version'<wbr/>: u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_<wbr/>api_version'<wbr/>: '2', 'verify': True, 'dns_api_version': '2', u'object_<wbr/>store_api_<wbr/>version'<wbr/>: u'1', 'username': 'adminapi', 'container_<wbr/>infra_api_<wbr/>version'<wbr/>: '1', 'loadbalancer_<wbr/>api_version'<wbr/>: '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_<wbr/>introspection_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'api_timeout': None, 'image_<wbr/>api_version'<wbr/>: '2', 'auth': {'user_<wbr/>domain_<wbr/>name': 'def...</p></div> <textarea style="width: 100%" rows="10">Hi @hongbin, I can't correct the problem, when I run on Compute Host:# docker exec --interactive zun-ca694bec-2e89-4a3f-b78b-ad21c9fe9353 /bin/bash it work perfectly But on the controller # openstack --debug appcontainer exec --interactive container /bin/sh it give Internal Server Error (HTTP 500) ___________________________________________________________________ #openstack --debug appcontainer exec --interactive container /bin/sh WARNING: Failed to import plugin clustering. START with options: --debug appcontainer exec --interactive container /bin/sh options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', aodh_endpoint='', application_credential_id='', application_credential_name='', application_credential_secret='***', auth_type='password', auth_url='http://controller:5000/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_api_version='2', os_beta_command=False, os_compute_api_version='', os_container_api_version='1.20', os_container_infra_api_version='1', os_data_processing_api_version='1.1', os_data_processing_url='', os_database_api_version='1', os_dns_api_version='2', os_identity_api_version='3', os_image_api_version='2', os_key_manager_api_version='1', os_loadbalancer_api_version='2.0', os_metrics_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_queues_api_version='2', os_volume_api_version='', os_workflow_api_version='2', passcode='', password='***', profile='', project_domain_id='', project_domain_name='default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='', remote_project_domain_id='', remote_project_domain_name='', remote_project_id='', remote_project_name='', roles='', service_provider='', service_provider_endpoint='', service_provider_entity_id='', system_scope='', timing=False, token='***', trust_id='', url='', user='', user_domain_id='', user_domain_name='default', user_id='', username='adminapi', verbose_level=3, verify=None) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} defaults: {u'auth_type': 'password', u'status': u'active', u'image_status_code_retries': 5, u'baremetal_introspection_status_code_retries': 5, 'api_timeout': None, 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, u'interface': u'public', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'baremetal_status_code_retries': 5, 'verify': True, 'cert': None, u'secgroup_source': u'neutron', u'object_store_api_version': u'1', u'disable_vendor_agent': {}} cloud cfg: {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'orchestration_api_version': '1', 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'container_api_version': '1.20', u'interface': 'public', u'disable_vendor_agent': {}} compute API version 2.1, cmd group openstack.compute.v2 network API version 2, cmd group openstack.network.v2 image API version 2, cmd group openstack.image.v2 volume API version 2, cmd group openstack.volume.v2 identity API version 3, cmd group openstack.identity.v3 object_store API version 1, cmd group openstack.object_store.v1 container API version 1.20, cmd group openstack.container.v1 messaging API version 2, cmd group openstack.messaging.v2 database API version 1, cmd group openstack.database.v1 data_processing API version 1.1, cmd group openstack.data_processing.v1 load_balancer API version 2.0, cmd group openstack.load_balancer.v2 neutronclient API version 2, cmd group openstack.neutronclient.v2 workflow_engine API version 2, cmd group openstack.workflow_engine.v2 container_infra API version 1, cmd group openstack.container_infra.v1 orchestration API version 1, cmd group openstack.orchestration.v1 dns API version 2, cmd group openstack.dns.v2 key_manager API version 1, cmd group openstack.key_manager.v1 metric API version 1, cmd group openstack.metric.v1 alarming API version 2, cmd group openstack.alarming.v2 Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} command: appcontainer exec -> zunclient.osc.v1.containers.ExecContainer (auth=True) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', 'additional_user_agent': [('osc-lib', '1.12.1')], u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} Using auth plugin: password Using parameters {'username': 'adminapi', 'project_name': 'admin', 'user_domain_name': 'default', 'auth_url': 'http://controller:5000/v3', 'password': '***', 'project_domain_name': 'default'} Get auth_ref REQ: curl -g -i -X GET http://controller:5000/v3 -H "Accept: application/json" -H "User-Agent: openstacksdk/0.28.0 keystoneauth1/3.14.0 python-requests/2.22.0 CPython/2.7.15rc1" Starting new HTTP connection (1): controller:5000 http://controller:5000 "GET /v3 HTTP/1.1" 200 250 RESP: [200] Connection: Keep-Alive Content-Length: 250 Content-Type: application/json Date: Sat, 01 Jun 2019 01:08:11 GMT Keep-Alive: timeout=5, max=100 Server: Apache/2.4.29 (Ubuntu) Vary: X-Auth-Token x-openstack-request-id: req-87b586e0-81e9-4adf-a6fb-cf7260f12490 RESP BODY: {"version": {"status": "stable", "updated": "2018-10-15T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.11", "links": [{"href": "http://controller:5000/v3/", "rel": "self"}]}} GET call to http://controller:5000/v3 used request id req-87b586e0-81e9-4adf-a6fb-cf7260f12490 Making authentication request to http://controller:5000/v3/auth/tokens http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a128816d407542ee836bd30d8c36484d"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": [{"url": "http://controller:8774/v2.1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": "http://controller:8774/v2.1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3b4eef8ae0d64c1396386a5557d50560"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": [{"url": "http://controller:9001/v2", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": "http://controller:9001/v2", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": [{"url": "http://controller:9292", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": [{"url": "http://controller:5000/v3", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": "http://controller:5000/v3", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": "keystone"}, {"endpoints": [{"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3827d192568f4b1e8b1901bdedbe1137"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "916be6147a2d4df293ebf375abd6fed2"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "56aebd20ee19475cae24f2e558de4b2d"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": [{"url": "http://controller:9511/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": "http://controller:9511/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", "name": "magnum"}, {"endpoints": [{"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "52504619399d467ba64a1f232129768c"}, {"url": "http://controller:8080/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": [{"url": "http://controller:8041", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": [{"url": "http://controller:8778", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": "ceilometer"}, {"endpoints": [{"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0145b180de764baabb3f8fc15e5e0883"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b6006479263b4faf81963f21f523e11d"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": [{"url": "http://controller:8000/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "7b4e560125674762b7d516605986f5aa"}, {"url": "http://controller:8000/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "04d221684116407981f9d00d177b077a"}, {"url": "http://controller:9517/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "848b8262069a4c4c91716c52644783d8"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "password_expires_at": null, "name": "adminapi", "id": "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": ["P3f93tEbQgqYfj-DWeDt0g"], "issued_at": "2019-06-01T01:08:12.000000Z"}} run(Namespace(command=[u'/bin/sh'], container=u'container', interactive=True)) take_action(Namespace(command=[u'/bin/sh'], container=u'container', interactive=True)) Instantiating zun client: <class 'zunclient.v1.client.Client'> Making authentication request to http://controller:5000/v3/auth/tokens http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a128816d407542ee836bd30d8c36484d"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": [{"url": "http://controller:8774/v2.1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": "http://controller:8774/v2.1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3b4eef8ae0d64c1396386a5557d50560"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": [{"url": "http://controller:9001/v2", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": "http://controller:9001/v2", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": [{"url": "http://controller:9292", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": [{"url": "http://controller:5000/v3", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": "http://controller:5000/v3", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": "keystone"}, {"endpoints": [{"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3827d192568f4b1e8b1901bdedbe1137"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "916be6147a2d4df293ebf375abd6fed2"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "56aebd20ee19475cae24f2e558de4b2d"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": [{"url": "http://controller:9511/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": "http://controller:9511/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", "name": "magnum"}, {"endpoints": [{"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "52504619399d467ba64a1f232129768c"}, {"url": "http://controller:8080/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": [{"url": "http://controller:8041", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": [{"url": "http://controller:8778", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": "ceilometer"}, {"endpoints": [{"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0145b180de764baabb3f8fc15e5e0883"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b6006479263b4faf81963f21f523e11d"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": [{"url": "http://controller:8000/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "7b4e560125674762b7d516605986f5aa"}, {"url": "http://controller:8000/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "04d221684116407981f9d00d177b077a"}, {"url": "http://controller:9517/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "848b8262069a4c4c91716c52644783d8"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "password_expires_at": null, "name": "adminapi", "id": "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": ["iGpHpBe5T-2Wl_gJML5UEA"], "issued_at": "2019-06-01T01:08:12.000000Z"}} REQ: curl -g -i -X POST http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: application/json" -H "OpenStack-API-Version: container 1.20" -H "User-Agent: None" -H "X-Auth-Token: {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" Starting new HTTP connection (1): controller:9517 http://controller:9517 "POST /v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True HTTP/1.1" 200 310 RESP: [200] Connection: keep-alive Content-Length: 310 Content-Type: application/json Date: Sat, 01 Jun 2019 01:08:13 GMT Openstack-Api-Maximum-Version: container 1.22 Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: req-060272d5-6505-4352-9e78-7e5e2761947f RESP BODY: {"output": null, "exec_id": "e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", "proxy_url": "ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", "exit_code": null} POST call to container for http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True used request id req-060272d5-6505-4352-9e78-7e5e2761947f connecting to: ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9 connected to container, press Enter to continue type ~. to disconnect REQ: curl -g -i -X POST http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: application/json" -H "OpenStack-API-Version: container 1.20" -H "User-Agent: None" -H "X-Auth-Token: {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" http://controller:9517 "POST /v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 HTTP/1.1" 500 261 RESP: [500] Connection: keep-alive Content-Length: 261 Content-Type: application/json Date: Sat, 01 Jun 2019 01:08:23 GMT Openstack-Api-Maximum-Version: container 1.22 Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: req-af129bd8-36af-4dbc-9446-54d181d5293b RESP BODY: {"errors": [{"status": 500, "code": "container.", "links": [], "title": "Internal Server Error", "detail": "Your request could not be handled because of a problem in the server. Error Correlation id is: 0c19d0a7-339b-47b8-b37a-d8db930a1d01", "request_id": ""}]} POST call to container for http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 used request id req-af129bd8-36af-4dbc-9446-54d181d5293b Internal Server Error (HTTP 500) (Request-ID: req-af129bd8-36af-4dbc-9446-54d181d5293b) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", line 537, in take_action websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 379, in do_exec wscls.handle_resize() File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 196, in handle_resize self.tty_resize(height=rows, width=cols) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 294, in tty_resize self.cs.containers.execute_resize(self.id, self.exec_id, width, height) File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 165, in execute_resize qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 128, in _action **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 367, in json_request resp = self._http_request(url, method, **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 350, in _http_request error_json.get('debuginfo'), method, url) InternalServerError: Internal Server Error (HTTP 500) (Request-ID: req-af129bd8-36af-4dbc-9446-54d181d5293b) clean_up ExecContainer: Internal Server Error (HTTP 500) (Request-ID: req-af129bd8-36af-4dbc-9446-54d181d5293b) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 136, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 281, in run result = self.run_subcommand(remainder) File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 176, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", line 537, in take_action websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 379, in do_exec wscls.handle_resize() File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 196, in handle_resize self.tty_resize(height=rows, width=cols) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 294, in tty_resize self.cs.containers.execute_resize(self.id, self.exec_id, width, height) File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 165, in execute_resize qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 128, in _action **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 367, in json_request resp = self._http_request(url, method, **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 350, in _http_request error_json.get('debuginfo'), method, url) InternalServerError: Internal Server Error (HTTP 500) (Request-ID: req-af129bd8-36af-4dbc-9446-54d181d5293b) END return value: 1 <a href="https://bugs.launchpad.net/zun/+bug/1830450/comments/9/+download">Download full text</a> (45.1 KiB) <div class="comment-text editable-message-text" itemprop="commentText"><p>Could you send me the Docker Daemon log. Please make sure the 'debug' mode<br/> is turned on:<br/> <a rel="nofollow" href="https://success.docker.com/article/how-do-i-enable-debug-logging-of-the-docker-daemon">https:/<wbr/>/success.<wbr/>docker.<wbr/>com/article/<wbr/>how-do-<wbr/>i-enable-<wbr/>debug-logging-<wbr/>of-the-<wbr/>docker-<wbr/>daemon</a></p> <p>On Fri, May 31, 2019 at 9:20 PM fatis <email address hidden> wrote:</p> <p><span class="foldable-quoted">> Hi @hongbin,<br/> > I can't correct the problem, when I run on Compute Host:# docker exec<br/> > --interactive zun-ca694bec-<wbr/>2e89-4a3f-<wbr/>b78b-ad21c9fe93<wbr/>53 /bin/bash it work<br/> > perfectly<br/> > But on the controller # openstack --debug appcontainer exec --interactive<br/> > container /bin/sh it give Internal Server Error (HTTP 500)<br/> > _______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>____<br/> > #openstack --debug appcontainer exec --interactive container /bin/sh<br/> > WARNING: Failed to import plugin clustering.<br/> > START with options: --debug appcontainer exec --interactive container<br/> > /bin/sh<br/> > options: Namespace(<wbr/>access_<wbr/>key='', access_<wbr/>secret=<wbr/>'***', access_token='***',<br/> > access_<wbr/>token_endpoint=<wbr/>'', access_<wbr/>token_type=<wbr/>'', aodh_endpoint='',<br/> > application_<wbr/>credential_<wbr/>id='', application_<wbr/>credential_<wbr/>name=''<wbr/>,<br/> > application_<wbr/>credential_<wbr/>secret=<wbr/>'***', auth_type=<wbr/>'password'<wbr/>, auth_url='<br/> > <a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, cacert=None, cert='', client_id='',<br/> > client_<wbr/>secret=<wbr/>'***', cloud='', code='', consumer_key='',<br/> > consumer_<wbr/>secret=<wbr/>'***', debug=True, default_<wbr/>domain=<wbr/>'default'<wbr/>,<br/> > default_<wbr/>domain_<wbr/>id='', default_<wbr/>domain_<wbr/>name=''<wbr/>, deferred_<wbr/>help=False,<br/> > discovery_<wbr/>endpoint=<wbr/>'', domain_id='', domain_name='', endpoint='',<br/> > identity_<wbr/>provider=<wbr/>'', identity_<wbr/>provider_<wbr/>url='', insecure=None,<br/> > interface='public', key='', log_file=None, openid_scope='',<br/> > os_alarming_<wbr/>api_version=<wbr/>'2', os_beta_<wbr/>command=<wbr/>False,<br/> > os_compute_<wbr/>api_version=<wbr/>'', os_container_<wbr/>api_version=<wbr/>'1.20',<br/> > os_container_<wbr/>infra_api_<wbr/>version=<wbr/>'1', os_data_<wbr/>processing_<wbr/>api_version=<wbr/>'1.1',<br/> > os_data_<wbr/>processing_<wbr/>url='', os_database_<wbr/>api_version=<wbr/>'1',<br/> > os_dns_<wbr/>api_version=<wbr/>'2', os_identity_<wbr/>api_version=<wbr/>'3',<br/> > os_image_<wbr/>api_version=<wbr/>'2', os_key_<wbr/>manager_<wbr/>api_version=<wbr/>'1',<br/> > os_loadbalancer<wbr/>_api_version=<wbr/>'2.0', os_metrics_<wbr/>api_version=<wbr/>'1',<br/> > os_network_<wbr/>api_version=<wbr/>'', os_object_<wbr/>api_version=<wbr/>'',<br/> > os_orchestratio<wbr/>n_api_version=<wbr/>'1', os_project_id=None, os_project_<wbr/>name=None,<br/> > os_queues_<wbr/>api_version=<wbr/>'2', os_volume_<wbr/>api_version=<wbr/>'',<br/> > os_workflow_<wbr/>api_version=<wbr/>'2', passcode='', password='***', profile='',<br/> > project_<wbr/>domain_<wbr/>id='', project_<wbr/>domain_<wbr/>name='default'<wbr/>, project_id='',<br/> > project_<wbr/>name='admin'<wbr/>, protocol='', redirect_uri='', region_name='',<br/> > remote_<wbr/>project_<wbr/>domain_<wbr/>id='', remote_<wbr/>project_<wbr/>domain_<wbr/>name=''<wbr/>,<br/> > remote_<wbr/>project_<wbr/>id='', remote_<wbr/>project_<wbr/>name=''<wbr/>, roles='',<br/> > service_<wbr/>provider=<wbr/>'', service_<wbr/>provider_<wbr/>endpoint=<wbr/>'',<br/> > service_<wbr/>provider_<wbr/>entity_<wbr/>id='', system_scope='', timing=False, token='***',<br/> > trust_id='', url='', user='', user_domain_id='',<br/> > user_domain_<wbr/>name='default'<wbr/>, user_id='', username=<wbr/>'adminapi'<wbr/>,<br/> > verbose_level=3, verify=None)<br/> > Auth plugin password selected<br/> > auth_config_hook(): {'auth_type': 'password', 'beta_command': False,<br/> > u'image_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'key': None, 'database_<wbr/>api_version'<wbr/>:<br/> > '1', 'metrics_<wbr/>api_version'<wbr/>: '1', 'data_processin<wbr/>g_api_version'<wbr/>: '1.1',<br/> > 'auth_url': '<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, u'network_<wbr/>api_version'<wbr/>: u'2',<br/> > u'message': u'', u'image_format': u'qcow2', 'networks': [],<br/> > 'queues_<wbr/>api_versio.<wbr/>.. <textarea style="width: 100%" rows="10">Could you send me the Docker Daemon log. Please make sure the 'debug' mode is turned on: https://success.docker.com/article/how-do-i-enable-debug-logging-of-the-docker-daemon On Fri, May 31, 2019 at 9:20 PM fatis <1830450@bugs.launchpad.net> wrote: > Hi @hongbin, > I can't correct the problem, when I run on Compute Host:# docker exec > --interactive zun-ca694bec-2e89-4a3f-b78b-ad21c9fe9353 /bin/bash it work > perfectly > But on the controller # openstack --debug appcontainer exec --interactive > container /bin/sh it give Internal Server Error (HTTP 500) > ___________________________________________________________________ > #openstack --debug appcontainer exec --interactive container /bin/sh > WARNING: Failed to import plugin clustering. > START with options: --debug appcontainer exec --interactive container > /bin/sh > options: Namespace(access_key='', access_secret='***', access_token='***', > access_token_endpoint='', access_token_type='', aodh_endpoint='', > application_credential_id='', application_credential_name='', > application_credential_secret='***', auth_type='password', auth_url=' > http://controller:5000/v3', cacert=None, cert='', client_id='', > client_secret='***', cloud='', code='', consumer_key='', > consumer_secret='***', debug=True, default_domain='default', > default_domain_id='', default_domain_name='', deferred_help=False, > discovery_endpoint='', domain_id='', domain_name='', endpoint='', > identity_provider='', identity_provider_url='', insecure=None, > interface='public', key='', log_file=None, openid_scope='', > os_alarming_api_version='2', os_beta_command=False, > os_compute_api_version='', os_container_api_version='1.20', > os_container_infra_api_version='1', os_data_processing_api_version='1.1', > os_data_processing_url='', os_database_api_version='1', > os_dns_api_version='2', os_identity_api_version='3', > os_image_api_version='2', os_key_manager_api_version='1', > os_loadbalancer_api_version='2.0', os_metrics_api_version='1', > os_network_api_version='', os_object_api_version='', > os_orchestration_api_version='1', os_project_id=None, os_project_name=None, > os_queues_api_version='2', os_volume_api_version='', > os_workflow_api_version='2', passcode='', password='***', profile='', > project_domain_id='', project_domain_name='default', project_id='', > project_name='admin', protocol='', redirect_uri='', region_name='', > remote_project_domain_id='', remote_project_domain_name='', > remote_project_id='', remote_project_name='', roles='', > service_provider='', service_provider_endpoint='', > service_provider_entity_id='', system_scope='', timing=False, token='***', > trust_id='', url='', user='', user_domain_id='', > user_domain_name='default', user_id='', username='adminapi', > verbose_level=3, verify=None) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > defaults: {u'auth_type': 'password', u'status': u'active', > u'image_status_code_retries': 5, > u'baremetal_introspection_status_code_retries': 5, 'api_timeout': None, > 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': > u'neutron', 'key': None, u'interface': u'public', u'network_api_version': > u'2', u'message': u'', u'image_format': u'qcow2', > u'baremetal_status_code_retries': 5, 'verify': True, 'cert': None, > u'secgroup_source': u'neutron', u'object_store_api_version': u'1', > u'disable_vendor_agent': {}} > cloud cfg: {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'orchestration_api_version': '1', > 'database_api_version': '1', 'metrics_api_version': '1', > 'data_processing_api_version': '1.1', 'auth_url': ' > http://controller:5000/v3', u'network_api_version': u'2', u'message': > u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, > u'floating_ip_source': u'neutron', 'key': None, 'timing': False, > 'password': '***', 'cacert': None, 'key_manager_api_version': '1', > 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', > 'identity_api_version': '3', 'deferred_help': False, 'cert': None, > u'secgroup_source': u'neutron', u'status': u'active', > 'alarming_api_version': '2', 'container_api_version': '1.20', u'interface': > 'public', u'disable_vendor_agent': {}} > compute API version 2.1, cmd group openstack.compute.v2 > network API version 2, cmd group openstack.network.v2 > image API version 2, cmd group openstack.image.v2 > volume API version 2, cmd group openstack.volume.v2 > identity API version 3, cmd group openstack.identity.v3 > object_store API version 1, cmd group openstack.object_store.v1 > container API version 1.20, cmd group openstack.container.v1 > messaging API version 2, cmd group openstack.messaging.v2 > database API version 1, cmd group openstack.database.v1 > data_processing API version 1.1, cmd group openstack.data_processing.v1 > load_balancer API version 2.0, cmd group openstack.load_balancer.v2 > neutronclient API version 2, cmd group openstack.neutronclient.v2 > workflow_engine API version 2, cmd group openstack.workflow_engine.v2 > container_infra API version 1, cmd group openstack.container_infra.v1 > orchestration API version 1, cmd group openstack.orchestration.v1 > dns API version 2, cmd group openstack.dns.v2 > key_manager API version 1, cmd group openstack.key_manager.v1 > metric API version 1, cmd group openstack.metric.v1 > alarming API version 2, cmd group openstack.alarming.v2 > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > command: appcontainer exec -> zunclient.osc.v1.containers.ExecContainer > (auth=True) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', 'additional_user_agent': > [('osc-lib', '1.12.1')], u'network_api_version': u'2', u'message': u'', > u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Using auth plugin: password > Using parameters {'username': 'adminapi', 'project_name': 'admin', > 'user_domain_name': 'default', 'auth_url': 'http://controller:5000/v3', > 'password': '***', 'project_domain_name': 'default'} > Get auth_ref > REQ: curl -g -i -X GET http://controller:5000/v3 -H "Accept: > application/json" -H "User-Agent: openstacksdk/0.28.0 keystoneauth1/3.14.0 > python-requests/2.22.0 CPython/2.7.15rc1" > Starting new HTTP connection (1): controller:5000 > http://controller:5000 "GET /v3 HTTP/1.1" 200 250 > RESP: [200] Connection: Keep-Alive Content-Length: 250 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:11 GMT Keep-Alive: timeout=5, > max=100 Server: Apache/2.4.29 (Ubuntu) Vary: X-Auth-Token > x-openstack-request-id: req-87b586e0-81e9-4adf-a6fb-cf7260f12490 > RESP BODY: {"version": {"status": "stable", "updated": > "2018-10-15T00:00:00Z", "media-types": [{"base": "application/json", > "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.11", > "links": [{"href": "http://controller:5000/v3/", "rel": "self"}]}} > GET call to http://controller:5000/v3 used request id > req-87b586e0-81e9-4adf-a6fb-cf7260f12490 > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["P3f93tEbQgqYfj-DWeDt0g"], "issued_at": "2019-06-01T01:08:12.000000Z"}} > run(Namespace(command=[u'/bin/sh'], container=u'container', > interactive=True)) > take_action(Namespace(command=[u'/bin/sh'], container=u'container', > interactive=True)) > Instantiating zun client: <class 'zunclient.v1.client.Client'> > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["iGpHpBe5T-2Wl_gJML5UEA"], "issued_at": "2019-06-01T01:08:12.000000Z"}} > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" > Starting new HTTP connection (1): controller:9517 > http://controller:9517 "POST > /v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > HTTP/1.1" 200 310 > RESP: [200] Connection: keep-alive Content-Length: 310 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:13 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-060272d5-6505-4352-9e78-7e5e2761947f > RESP BODY: {"output": null, "exec_id": > "e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", > "proxy_url": > "ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", > "exit_code": null} > POST call to container for > http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > used request id req-060272d5-6505-4352-9e78-7e5e2761947f > connecting to: > ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9 > connected to container, press Enter to continue > type ~. to disconnect > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" > http://controller:9517 "POST > /v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > HTTP/1.1" 500 261 > RESP: [500] Connection: keep-alive Content-Length: 261 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:23 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-af129bd8-36af-4dbc-9446-54d181d5293b > RESP BODY: {"errors": [{"status": 500, "code": "container.", "links": [], > "title": "Internal Server Error", "detail": "Your request could not be > handled because of a problem in the server. Error Correlation id is: > 0c19d0a7-339b-47b8-b37a-d8db930a1d01", "request_id": ""}]} > POST call to container for > http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > used request id req-af129bd8-36af-4dbc-9446-54d181d5293b > Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > clean_up ExecContainer: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 136, in run > ret_val = super(OpenStackShell, self).run(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 281, in > result = self.run_subcommand(remainder) > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 176, in run_subcommand > ret_value = super(OpenStackShell, self).run_subcommand(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > END return value: 1 > You received this bug notification because you are subscribed to Zun. > Matching subscriptions: zun > https://bugs.launchpad.net/bugs/1830450 > Title: > openstack appcontainer exec --interactive > Status in Zun: > Bug description: > I have this error when I exec intercatve container > # openstack appcontainer exec --interactive container /bin/sh > HTTPConnectionPool(host='130.127.133.20', port=2375): Max retries > exceeded with url: > /v1.35/exec/808ebed1841484c028d0f0684f7637fe28849592125607c55173155ca0f47668/start > (Caused by NewConnectionError('<urllib3.connection.HTTPConnection > object at 0x7f44ae9ef550>: Failed to establish a new connection: > [Errno 111] Connection refused',)) > /etc/systemd/system/docker.service.d/docker.conf #my compute node > [Service] > ExecStart= > ExecStart=/usr/bin/dockerd --group zun -H tcp://compute-1:2375 -H > unix:///var/run/docker.sock --cluster-store etcd://controller:2379 > To manage notifications about this bug go to: > https://bugs.launchpad.net/zun/+bug/1830450/+subscriptions <a href="https://bugs.launchpad.net/zun/+bug/1830450/comments/10/+download">Download full text</a> (45.0 KiB) <div class="comment-text editable-message-text" itemprop="commentText"><p>In addition, it would be great if you can paste me your docker info:</p> <p>  $ docker info</p> <p>On Fri, May 31, 2019 at 9:20 PM fatis <email address hidden> wrote:</p> <p><span class="foldable-quoted">> Hi @hongbin,<br/> > I can't correct the problem, when I run on Compute Host:# docker exec<br/> > --interactive zun-ca694bec-<wbr/>2e89-4a3f-<wbr/>b78b-ad21c9fe93<wbr/>53 /bin/bash it work<br/> > perfectly<br/> > But on the controller # openstack --debug appcontainer exec --interactive<br/> > container /bin/sh it give Internal Server Error (HTTP 500)<br/> > _______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>____<br/> > #openstack --debug appcontainer exec --interactive container /bin/sh<br/> > WARNING: Failed to import plugin clustering.<br/> > START with options: --debug appcontainer exec --interactive container<br/> > /bin/sh<br/> > options: Namespace(<wbr/>access_<wbr/>key='', access_<wbr/>secret=<wbr/>'***', access_token='***',<br/> > access_<wbr/>token_endpoint=<wbr/>'', access_<wbr/>token_type=<wbr/>'', aodh_endpoint='',<br/> > application_<wbr/>credential_<wbr/>id='', application_<wbr/>credential_<wbr/>name=''<wbr/>,<br/> > application_<wbr/>credential_<wbr/>secret=<wbr/>'***', auth_type=<wbr/>'password'<wbr/>, auth_url='<br/> > <a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, cacert=None, cert='', client_id='',<br/> > client_<wbr/>secret=<wbr/>'***', cloud='', code='', consumer_key='',<br/> > consumer_<wbr/>secret=<wbr/>'***', debug=True, default_<wbr/>domain=<wbr/>'default'<wbr/>,<br/> > default_<wbr/>domain_<wbr/>id='', default_<wbr/>domain_<wbr/>name=''<wbr/>, deferred_<wbr/>help=False,<br/> > discovery_<wbr/>endpoint=<wbr/>'', domain_id='', domain_name='', endpoint='',<br/> > identity_<wbr/>provider=<wbr/>'', identity_<wbr/>provider_<wbr/>url='', insecure=None,<br/> > interface='public', key='', log_file=None, openid_scope='',<br/> > os_alarming_<wbr/>api_version=<wbr/>'2', os_beta_<wbr/>command=<wbr/>False,<br/> > os_compute_<wbr/>api_version=<wbr/>'', os_container_<wbr/>api_version=<wbr/>'1.20',<br/> > os_container_<wbr/>infra_api_<wbr/>version=<wbr/>'1', os_data_<wbr/>processing_<wbr/>api_version=<wbr/>'1.1',<br/> > os_data_<wbr/>processing_<wbr/>url='', os_database_<wbr/>api_version=<wbr/>'1',<br/> > os_dns_<wbr/>api_version=<wbr/>'2', os_identity_<wbr/>api_version=<wbr/>'3',<br/> > os_image_<wbr/>api_version=<wbr/>'2', os_key_<wbr/>manager_<wbr/>api_version=<wbr/>'1',<br/> > os_loadbalancer<wbr/>_api_version=<wbr/>'2.0', os_metrics_<wbr/>api_version=<wbr/>'1',<br/> > os_network_<wbr/>api_version=<wbr/>'', os_object_<wbr/>api_version=<wbr/>'',<br/> > os_orchestratio<wbr/>n_api_version=<wbr/>'1', os_project_id=None, os_project_<wbr/>name=None,<br/> > os_queues_<wbr/>api_version=<wbr/>'2', os_volume_<wbr/>api_version=<wbr/>'',<br/> > os_workflow_<wbr/>api_version=<wbr/>'2', passcode='', password='***', profile='',<br/> > project_<wbr/>domain_<wbr/>id='', project_<wbr/>domain_<wbr/>name='default'<wbr/>, project_id='',<br/> > project_<wbr/>name='admin'<wbr/>, protocol='', redirect_uri='', region_name='',<br/> > remote_<wbr/>project_<wbr/>domain_<wbr/>id='', remote_<wbr/>project_<wbr/>domain_<wbr/>name=''<wbr/>,<br/> > remote_<wbr/>project_<wbr/>id='', remote_<wbr/>project_<wbr/>name=''<wbr/>, roles='',<br/> > service_<wbr/>provider=<wbr/>'', service_<wbr/>provider_<wbr/>endpoint=<wbr/>'',<br/> > service_<wbr/>provider_<wbr/>entity_<wbr/>id='', system_scope='', timing=False, token='***',<br/> > trust_id='', url='', user='', user_domain_id='',<br/> > user_domain_<wbr/>name='default'<wbr/>, user_id='', username=<wbr/>'adminapi'<wbr/>,<br/> > verbose_level=3, verify=None)<br/> > Auth plugin password selected<br/> > auth_config_hook(): {'auth_type': 'password', 'beta_command': False,<br/> > u'image_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'key': None, 'database_<wbr/>api_version'<wbr/>:<br/> > '1', 'metrics_<wbr/>api_version'<wbr/>: '1', 'data_processin<wbr/>g_api_version'<wbr/>: '1.1',<br/> > 'auth_url': '<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, u'network_<wbr/>api_version'<wbr/>: u'2',<br/> > u'message': u'', u'image_format': u'qcow2', 'networks': [],<br/> > 'queues_<wbr/>api_version'<wbr/>: '2', 'verify': True, 'dns_api_version': '2',<br/> > u'object_<wbr/>store_api_<wbr/>version'<wbr/>: u'1', 'us... > I can't correct the problem, when I run on Compute Host:# docker exec > --interactive zun-ca694bec-2e89-4a3f-b78b-ad21c9fe9353 /bin/bash it work > perfectly > But on the controller # openstack --debug appcontainer exec --interactive > container /bin/sh it give Internal Server Error (HTTP 500) > ___________________________________________________________________ > #openstack --debug appcontainer exec --interactive container /bin/sh > WARNING: Failed to import plugin clustering. > START with options: --debug appcontainer exec --interactive container > /bin/sh > options: Namespace(access_key='', access_secret='***', access_token='***', > access_token_endpoint='', access_token_type='', aodh_endpoint='', > application_credential_id='', application_credential_name='', > application_credential_secret='***', auth_type='password', auth_url=' > http://controller:5000/v3', cacert=None, cert='', client_id='', > client_secret='***', cloud='', code='', consumer_key='', > consumer_secret='***', debug=True, default_domain='default', > default_domain_id='', default_domain_name='', deferred_help=False, > discovery_endpoint='', domain_id='', domain_name='', endpoint='', > identity_provider='', identity_provider_url='', insecure=None, > interface='public', key='', log_file=None, openid_scope='', > os_alarming_api_version='2', os_beta_command=False, > os_compute_api_version='', os_container_api_version='1.20', > os_container_infra_api_version='1', os_data_processing_api_version='1.1', > os_data_processing_url='', os_database_api_version='1', > os_dns_api_version='2', os_identity_api_version='3', > os_image_api_version='2', os_key_manager_api_version='1', > os_loadbalancer_api_version='2.0', os_metrics_api_version='1', > os_network_api_version='', os_object_api_version='', > os_orchestration_api_version='1', os_project_id=None, os_project_name=None, > os_queues_api_version='2', os_volume_api_version='', > os_workflow_api_version='2', passcode='', password='***', profile='', > project_domain_id='', project_domain_name='default', project_id='', > project_name='admin', protocol='', redirect_uri='', region_name='', > remote_project_domain_id='', remote_project_domain_name='', > remote_project_id='', remote_project_name='', roles='', > service_provider='', service_provider_endpoint='', > service_provider_entity_id='', system_scope='', timing=False, token='***', > trust_id='', url='', user='', user_domain_id='', > user_domain_name='default', user_id='', username='adminapi', > verbose_level=3, verify=None) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > defaults: {u'auth_type': 'password', u'status': u'active', > u'image_status_code_retries': 5, > u'baremetal_introspection_status_code_retries': 5, 'api_timeout': None, > 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': > u'neutron', 'key': None, u'interface': u'public', u'network_api_version': > u'2', u'message': u'', u'image_format': u'qcow2', > u'baremetal_status_code_retries': 5, 'verify': True, 'cert': None, > u'secgroup_source': u'neutron', u'object_store_api_version': u'1', > u'disable_vendor_agent': {}} > cloud cfg: {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'orchestration_api_version': '1', > 'database_api_version': '1', 'metrics_api_version': '1', > 'data_processing_api_version': '1.1', 'auth_url': ' > http://controller:5000/v3', u'network_api_version': u'2', u'message': > u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, > u'floating_ip_source': u'neutron', 'key': None, 'timing': False, > 'password': '***', 'cacert': None, 'key_manager_api_version': '1', > 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', > 'identity_api_version': '3', 'deferred_help': False, 'cert': None, > u'secgroup_source': u'neutron', u'status': u'active', > 'alarming_api_version': '2', 'container_api_version': '1.20', u'interface': > 'public', u'disable_vendor_agent': {}} > compute API version 2.1, cmd group openstack.compute.v2 > network API version 2, cmd group openstack.network.v2 > image API version 2, cmd group openstack.image.v2 > volume API version 2, cmd group openstack.volume.v2 > identity API version 3, cmd group openstack.identity.v3 > object_store API version 1, cmd group openstack.object_store.v1 > container API version 1.20, cmd group openstack.container.v1 > messaging API version 2, cmd group openstack.messaging.v2 > database API version 1, cmd group openstack.database.v1 > data_processing API version 1.1, cmd group openstack.data_processing.v1 > load_balancer API version 2.0, cmd group openstack.load_balancer.v2 > neutronclient API version 2, cmd group openstack.neutronclient.v2 > workflow_engine API version 2, cmd group openstack.workflow_engine.v2 > container_infra API version 1, cmd group openstack.container_infra.v1 > orchestration API version 1, cmd group openstack.orchestration.v1 > dns API version 2, cmd group openstack.dns.v2 > key_manager API version 1, cmd group openstack.key_manager.v1 > metric API version 1, cmd group openstack.metric.v1 > alarming API version 2, cmd group openstack.alarming.v2 > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > command: appcontainer exec -> zunclient.osc.v1.containers.ExecContainer > (auth=True) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', 'additional_user_agent': > [('osc-lib', '1.12.1')], u'network_api_version': u'2', u'message': u'', > u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Using auth plugin: password > Using parameters {'username': 'adminapi', 'project_name': 'admin', > 'user_domain_name': 'default', 'auth_url': 'http://controller:5000/v3', > 'password': '***', 'project_domain_name': 'default'} > Get auth_ref > REQ: curl -g -i -X GET http://controller:5000/v3 -H "Accept: > application/json" -H "User-Agent: openstacksdk/0.28.0 keystoneauth1/3.14.0 > python-requests/2.22.0 CPython/2.7.15rc1" > Starting new HTTP connection (1): controller:5000 > http://controller:5000 "GET /v3 HTTP/1.1" 200 250 > RESP: [200] Connection: Keep-Alive Content-Length: 250 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:11 GMT Keep-Alive: timeout=5, > max=100 Server: Apache/2.4.29 (Ubuntu) Vary: X-Auth-Token > x-openstack-request-id: req-87b586e0-81e9-4adf-a6fb-cf7260f12490 > RESP BODY: {"version": {"status": "stable", "updated": > "2018-10-15T00:00:00Z", "media-types": [{"base": "application/json", > "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.11", > "links": [{"href": "http://controller:5000/v3/", "rel": "self"}]}} > GET call to http://controller:5000/v3 used request id > req-87b586e0-81e9-4adf-a6fb-cf7260f12490 > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["P3f93tEbQgqYfj-DWeDt0g"], "issued_at": "2019-06-01T01:08:12.000000Z"}} > run(Namespace(command=[u'/bin/sh'], container=u'container', > interactive=True)) > take_action(Namespace(command=[u'/bin/sh'], container=u'container', > interactive=True)) > Instantiating zun client: <class 'zunclient.v1.client.Client'> > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["iGpHpBe5T-2Wl_gJML5UEA"], "issued_at": "2019-06-01T01:08:12.000000Z"}} > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" > Starting new HTTP connection (1): controller:9517 > http://controller:9517 "POST > /v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > HTTP/1.1" 200 310 > RESP: [200] Connection: keep-alive Content-Length: 310 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:13 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-060272d5-6505-4352-9e78-7e5e2761947f > RESP BODY: {"output": null, "exec_id": > "e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", > "proxy_url": > "ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", > "exit_code": null} > POST call to container for > http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > used request id req-060272d5-6505-4352-9e78-7e5e2761947f > connecting to: > ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9 > connected to container, press Enter to continue > type ~. to disconnect > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" > http://controller:9517 "POST > /v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > HTTP/1.1" 500 261 > RESP: [500] Connection: keep-alive Content-Length: 261 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:23 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-af129bd8-36af-4dbc-9446-54d181d5293b > RESP BODY: {"errors": [{"status": 500, "code": "container.", "links": [], > "title": "Internal Server Error", "detail": "Your request could not be > handled because of a problem in the server. Error Correlation id is: > 0c19d0a7-339b-47b8-b37a-d8db930a1d01", "request_id": ""}]} > POST call to container for > http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > used request id req-af129bd8-36af-4dbc-9446-54d181d5293b > Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > clean_up ExecContainer: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 136, in run > ret_val = super(OpenStackShell, self).run(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 281, in > result = self.run_subcommand(remainder) > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 176, in run_subcommand > ret_value = super(OpenStackShell, self).run_subcommand(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > END return value: 1 > You received this bug notification because you are subscribed to Zun. > Matching subscriptions: zun > https://bugs.launchpad.net/bugs/1830450 > Title: > openstack appcontainer exec --interactive > Status in Zun: > Bug description: > I have this error when I exec intercatve container > # openstack appcontainer exec --interactive container /bin/sh > HTTPConnectionPool(host='130.127.133.20', port=2375): Max retries > exceeded with url: > /v1.35/exec/808ebed1841484c028d0f0684f7637fe28849592125607c55173155ca0f47668/start > (Caused by NewConnectionError('<urllib3.connection.HTTPConnection > object at 0x7f44ae9ef550>: Failed to establish a new connection: > [Errno 111] Connection refused',)) > /etc/systemd/system/docker.service.d/docker.conf #my compute node > [Service] > ExecStart= > ExecStart=/usr/bin/dockerd --group zun -H tcp://compute-1:2375 -H > unix:///var/run/docker.sock --cluster-store etcd://controller:2379 > To manage notifications about this bug go to: > https://bugs.launchpad.net/zun/+bug/1830450/+subscriptions <p># journalctl -u docker.service</p> <p># cat docker.log<br/> -- Logs begin at Fri 2019-05-24 19:45:41 EDT, end at Sat 2019-06-01 13:21:34 EDT. --<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us systemd[1]: Starting Docker Application Container Engine...<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.858369071<wbr/>-04:00" level=info msg="systemd-<wbr/>resolved is running, so using resolvconf: /run/systemd/<wbr/>resolve/<wbr/>resolv.<wbr/>conf"<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903362201<wbr/>-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903397841<wbr/>-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903502237<wbr/>-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903518953<wbr/>-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903506998<wbr/>-04:00" level=info msg="ccResolver<wbr/>Wrapper: sending new addresses to cc: [{unix:<wbr/>///run/<wbr/>containerd/<wbr/>containerd.<wbr/>sock 0 <nil>}]" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903555369<wbr/>-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903612640<wbr/>-04:00" level=info msg="pickfirstB<wbr/>alancer: HandleSubConnSt<wbr/>ateChange: 0xc420822180, CONNECTING" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903684138<wbr/>-04:00" level=info msg="ccResolver<wbr/>Wrapper: sending new addresses to cc: [{unix:<wbr/>///run/<wbr/>containerd/<wbr/>containerd.<wbr/>sock 0 <nil>}]" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903743833<wbr/>-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903812509<wbr/>-04:00" level=info msg="pickfirstB<wbr/>alancer: HandleSubConnSt<wbr/>ateChange: 0xc4207a2150, CONNECTING" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903835625<wbr/>-04:00" level=info msg="pickfirstB<wbr/>alancer: HandleSubConnSt<wbr/>ateChange: 0xc420822180, READY" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903833860<wbr/>-04:00" level=info msg="blockingPi<wbr/>cker: the picked transport is not ready, loop back to repick" module=grpc<br/> May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.904000665<wbr/>-04:00" level=info msg="pickfirstB<wbr/>alancer: H...</p></div> # cat docker.log -- Logs begin at Fri 2019-05-24 19:45:41 EDT, end at Sat 2019-06-01 13:21:34 EDT. -- May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Starting Docker Application Container Engine... May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.858369071-04:00" level=info msg="systemd-resolved is running, so using resolvconf: /run/systemd/resolve/resolv.conf" May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903362201-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903397841-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903502237-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903518953-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903506998-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903555369-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903612640-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420822180, CONNECTING" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903684138-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903743833-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903812509-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207a2150, CONNECTING" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903835625-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420822180, READY" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.903833860-04:00" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:49.904000665-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207a2150, READY" module=grpc May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:50.104124302-04:00" level=info msg="Graph migration to content-addressability took 0.00 seconds" May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:50.104474636-04:00" level=warning msg="Your kernel does not support swap memory limit" May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:50.104591093-04:00" level=warning msg="Your kernel does not support cgroup rt period" May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:50.104618391-04:00" level=warning msg="Your kernel does not support cgroup rt runtime" May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:50.105250169-04:00" level=info msg="Loading containers: start." May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:50.422816653-04:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address" May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:50.559727165-04:00" level=info msg="Loading containers: done." May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:51.208912044-04:00" level=info msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:51.209074335-04:00" level=info msg="Daemon has completed initialization" May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:51.496957523-04:00" level=info msg="API listen on /var/run/docker.sock" May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Started Docker Application Container Engine. May 24 19:55:59 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T19:55:59.710302568-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Stopping Docker Application Container Engine... May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T20:01:30.789804236-04:00" level=info msg="Processing signal 'terminated'" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[8383]: time="2019-05-24T20:01:30.790693612-04:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Stopped Docker Application Container Engine. May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Starting Docker Application Container Engine... May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.861161946-04:00" level=info msg="systemd-resolved is running, so using resolvconf: /run/systemd/resolve/resolv.conf" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862590017-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862630186-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862683154-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862696670-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862760787-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862837705-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862779304-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862927302-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.862939509-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42070a200, CONNECTING" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.863017496-04:00" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.863015360-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c7610, CONNECTING" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.863224369-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42070a200, READY" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.863254133-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c7610, READY" module=grpc May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.864817636-04:00" level=info msg="[graphdriver] using prior storage driver: overlay2" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.893527775-04:00" level=info msg="Graph migration to content-addressability took 0.00 seconds" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.893819559-04:00" level=warning msg="Your kernel does not support swap memory limit" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.893876995-04:00" level=warning msg="Your kernel does not support cgroup rt period" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.893893904-04:00" level=warning msg="Your kernel does not support cgroup rt runtime" May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:30.894433695-04:00" level=info msg="Loading containers: start." May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:31.133946971-04:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address" May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:31.241225541-04:00" level=info msg="Loading containers: done." May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:31.471177743-04:00" level=info msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:31.471267752-04:00" level=info msg="Daemon has completed initialization" May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:01:31.527275562-04:00" level=info msg="API listen on /var/run/docker.sock" May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Started Docker Application Container Engine. May 24 20:17:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: docker.service: Current command vanished from the unit file, execution of the command list won't be resumed. May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Stopping Docker Application Container Engine... May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:17:37.489068376-04:00" level=info msg="Processing signal 'terminated'" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:17:37.489933441-04:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[11329]: time="2019-05-24T20:17:37.490408324-04:00" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Stopped Docker Application Container Engine. May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Starting Docker Application Container Engine... May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.550025542-04:00" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.561837754-04:00" level=info msg="systemd-resolved is running, so using resolvconf: /run/systemd/resolve/resolv.conf" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.562840608-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.562874490-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.562931637-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.562945759-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563067876-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563072320-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563147871-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563102368-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563241106-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c6700, CONNECTING" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563300921-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42080a1a0, CONNECTING" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563476574-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42080a1a0, READY" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.563477762-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c6700, READY" module=grpc May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.564952835-04:00" level=info msg="[graphdriver] using prior storage driver: overlay2" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.602153582-04:00" level=info msg="Graph migration to content-addressability took 0.00 seconds" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.602468861-04:00" level=warning msg="Your kernel does not support swap memory limit" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.602531493-04:00" level=warning msg="Your kernel does not support cgroup rt period" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.602562008-04:00" level=warning msg="Your kernel does not support cgroup rt runtime" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.603144892-04:00" level=info msg="Loading containers: start." May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.850881037-04:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address" May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:37.962900503-04:00" level=info msg="Loading containers: done." May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:38.210057261-04:00" level=info msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:38.210152641-04:00" level=info msg="Daemon has completed initialization" May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:38.276678580-04:00" level=info msg="API listen on /var/run/docker.sock" May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-24T20:17:38.276721864-04:00" level=info msg="API listen on 192.168.0.5:2375" May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Started Docker Application Container Engine. May 25 11:56:43 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-25T11:56:43.488871946-04:00" level=warning msg="Your kernel does not support swap limit capabilities,or the cgroup is not mounted. Memory limited without swap." May 25 12:01:42 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-25T12:01:42.586332098-04:00" level=info msg="Container 50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0 failed to exit within 10 seconds of signal 15 - using the force" May 25 12:01:42 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-25T12:01:42.783250781-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 25 13:07:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-25T13:07:12.249476332-04:00" level=error msg="Not continuing with pull after error: errors:\ndenied: requested access to the resource is denied\nunauthorized: authentication required\n" May 25 13:07:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-25T13:07:12.249524106-04:00" level=info msg="Ignoring extra error returned from registry: unauthorized: authentication required" May 28 09:28:01 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:28:01.044559035-04:00" level=error msg="stream copy error: reading from a closed fifo" May 28 09:28:01 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:28:01.044587325-04:00" level=error msg="stream copy error: reading from a closed fifo" May 28 09:28:01 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:28:01.115233194-04:00" level=error msg="Error running exec 9b266f33f7e6108d406268a63859eb68e144c39319c3c82f9a73c9cae8e8217c in container: OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused \"exec: \\\"/bin/bash\\\": stat /bin/bash: no such file or directory\": unknown" May 28 09:33:08 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:33:08.563751788-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 28 09:44:13 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:44:13.374623021-04:00" level=info msg="Container 50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0 failed to exit within 10 seconds of signal 15 - using the force" May 28 09:44:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:44:23.414223215-04:00" level=info msg="Container 50fc977abf05 failed to exit within 10 seconds of kill - trying direct SIGKILL" May 28 09:47:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:47:26.078689928-04:00" level=info msg="Container 50fc977abf05 failed to exit within 10 seconds of kill - trying direct SIGKILL" May 28 09:47:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:47:29.122526145-04:00" level=info msg="Container 50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0 failed to exit within 3 seconds of signal 15 - using the force" May 28 09:47:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:47:39.166464318-04:00" level=info msg="Container 50fc977abf05 failed to exit within 10 seconds of kill - trying direct SIGKILL" May 28 09:53:43 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-28T09:53:43.256877588-04:00" level=warning msg="Your kernel does not support swap limit capabilities,or the cgroup is not mounted. Memory limited without swap." May 29 10:32:57 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-29T10:32:57.052367435-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 29 10:33:08 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-29T10:33:08.319816062-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 29 10:33:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-29T10:33:12.932672654-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 29 12:48:13 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-29T12:48:13.998694211-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 29 13:15:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-29T13:15:26.696615580-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 30 09:05:45 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:05:45.901276733-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 30 09:09:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:09:12.955469478-04:00" level=error msg="Not continuing with pull after error: errors:\ndenied: requested access to the resource is denied\nunauthorized: authentication required\n" May 30 09:09:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:09:12.955517276-04:00" level=info msg="Ignoring extra error returned from registry: unauthorized: authentication required" May 30 09:12:53 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:12:53.009059583-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 30 09:36:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: docker.service: Current command vanished from the unit file, execution of the command list won't be resumed. May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Stopping Docker Application Container Engine... May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:36:23.299783310-04:00" level=info msg="Processing signal 'terminated'" May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:36:23.510044573-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:36:23.886396108-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[12715]: time="2019-05-30T09:36:29.255297521-04:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Stopped Docker Application Container Engine. May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Starting Docker Application Container Engine... May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.318196152-04:00" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.328938560-04:00" level=info msg="systemd-resolved is running, so using resolvconf: /run/systemd/resolve/resolv.conf" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.330861153-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.330967939-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.331099437-04:00" level=info msg="parsed scheme: \"unix\"" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.331123399-04:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.331471091-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.331610299-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.331537943-04:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.331788095-04:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.332117669-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420ba4060, CONNECTING" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.332131455-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420b78180, CONNECTING" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.332146022-04:00" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.332427173-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420ba4060, READY" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.332686847-04:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420b78180, READY" module=grpc May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.334234395-04:00" level=info msg="[graphdriver] using prior storage driver: overlay2" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.369888614-04:00" level=info msg="Graph migration to content-addressability took 0.00 seconds" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.370259821-04:00" level=warning msg="Your kernel does not support swap memory limit" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.370342699-04:00" level=warning msg="Your kernel does not support cgroup rt period" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.370376461-04:00" level=warning msg="Your kernel does not support cgroup rt runtime" May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:29.371004766-04:00" level=info msg="Loading containers: start." May 30 09:36:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:30.057651750-04:00" level=info msg="Removing stale sandbox a463e6c4b0b64ff925c30ff12c76c7e29db2b332c26fdc7649214918ed7e4f28 (50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0)" May 30 09:36:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:30.234660162-04:00" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 63699a212dae47fecdc8078df2ac310513e27cd98531d157fe5b75fd538fd66e fa7cbce00674806f4c3bf3c55c225d4c48b0670c1906e1c99d3ce9d540f826cf], retrying...." May 30 09:36:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:31.950540494-04:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address" May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:32.091239729-04:00" level=info msg="Loading containers: done." May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:32.630717553-04:00" level=info msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:32.630829877-04:00" level=info msg="Daemon has completed initialization" May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:32.707556989-04:00" level=info msg="API listen on 192.168.0.5:2375" May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-30T09:36:32.707584381-04:00" level=info msg="API listen on /var/run/docker.sock" May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: Started Docker Application Container Engine. May 31 10:45:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-31T10:45:38.594435640-04:00" level=info msg="Container 655b095307bdde41a75aad3ee654e71985fd6bad0e4c015ed9b656ff5d1cd920 failed to exit within 10 seconds of signal 15 - using the force" May 31 10:45:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-05-31T10:45:38.823583167-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Jun 01 13:13:54 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:13:54.770951837-04:00" level=info msg="Got signal to reload configuration, reloading from: /etc/docker/daemon.json" Jun 01 13:13:54 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:13:54.771747636-04:00" level=info msg="Reloaded configuration: {\"mtu\":1500,\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"zun\",\"deprecated-key-path\":\"/etc/docker/key.json\",\"cluster-store\":\"etcd://192.168.0.1:2379\",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"shutdown-timeout\":15,\"debug\":true,\"hosts\":[\"tcp://192.168.0.5:2375\",\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"log-driver\":\"json-file\",\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"default-address-pools\":{},\"network-control-plane-mtu\":1500,\"disable-legacy-registry\":true,\"experimental\":false,\"containerd\":\"/run/containerd/containerd.sock\",\"builder\":{\"GC\":{}},\"runtimes\":{\"runc\":{\"path\":\"runc\"}},\"default-runtime\":\"runc\",\"oom-score-adjust\":-500,\"default-shm-size\":67108864,\"default-ipc-mode\":\"shareable\",\"resolv-conf\":\"/run/systemd/resolve/resolv.conf\"}" Jun 01 13:14:00 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:00.631429844-04:00" level=debug msg="Calling GET /_ping" Jun 01 13:14:00 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:00.632011830-04:00" level=debug msg="Calling GET /v1.39/info" Jun 01 13:14:00 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:00.673263619-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:14:20 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:20.504062855-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.664901706-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.689117419-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.692574836-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.696289343-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.698938875-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.701271804-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.704400654-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:27.727719445-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:14:46 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:46.430200484-04:00" level=debug msg="Calling GET /_ping" Jun 01 13:14:46 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:14:46.431411964-04:00" level=debug msg="Calling GET /v1.39/containers/json" Jun 01 13:15:22 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:22.504416027-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.668542803-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.692771341-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.696259087-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.699904113-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.702294246-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.704802908-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.707913866-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:29.730612269-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:15:35 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:35.717889451-04:00" level=debug msg="Calling GET /_ping" Jun 01 13:15:35 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:35.718336063-04:00" level=debug msg="Calling GET /v1.39/info" Jun 01 13:15:35 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:15:35.760989788-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:16:24 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:24.503942833-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.664961401-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.689205294-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.692546135-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.696252604-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.698610508-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.701117700-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.704261463-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:16:30.726543208-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:17:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:26.504347290-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.664386055-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.688535243-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.691917116-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.695442614-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.697782366-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.700181957-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.703130620-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:17:32.724627595-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:18:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:26.531605753-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.142981125-04:00" level=debug msg="Calling GET /_ping" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.143415870-04:00" level=debug msg="Calling GET /v1.39/containers/zun-9bd314af-a8e0-4aa0-96d7-eb69345d2e79/json" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.145724199-04:00" level=debug msg="Calling POST /v1.39/containers/zun-9bd314af-a8e0-4aa0-96d7-eb69345d2e79/exec" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.145855963-04:00" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":true,\"AttachStdout\":true,\"Cmd\":[\"/bin/sh\"],\"Detach\":false,\"DetachKeys\":\"\",\"Env\":null,\"Privileged\":false,\"Tty\":false,\"User\":\"\",\"WorkingDir\":\"\"}" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.146333778-04:00" level=debug msg="Calling POST /v1.39/exec/e9c97eb44b73b258b165a01e6b218305e82c8a39187872fa26376abd75dff518/start" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.146407864-04:00" level=debug msg="form data: {\"Detach\":false,\"Tty\":false}" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.146513845-04:00" level=debug msg="starting exec command e9c97eb44b73b258b165a01e6b218305e82c8a39187872fa26376abd75dff518 in container cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.148885264-04:00" level=debug msg="attach: stdout: begin" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.148910971-04:00" level=debug msg="attach: stdin: begin" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.148940708-04:00" level=debug msg="attach: stderr: begin" Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:28.300965501-04:00" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/exec-started Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.664519878-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.688368815-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.691653911-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.695201603-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.697493585-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.699750823-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.702692702-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:33.724664601-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:39.848349267-04:00" level=debug msg="Closing buffered stdin pipe" Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:39.848415447-04:00" level=debug msg="attach: stdin: end" Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:39.935555994-04:00" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/exit Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:39.935648817-04:00" level=debug msg="attach: stdout: end" Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:39.935712539-04:00" level=debug msg="attach: stderr: end" Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:18:39.935739458-04:00" level=debug msg="attach done" Jun 01 13:19:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:26.559287617-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.672470236-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.696243049-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.699493238-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.703072526-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.705361140-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.707579027-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.710513180-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:19:33.732635018-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:20:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:26.586968988-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.665040671-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.689059381-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.692514730-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.696117352-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.698505933-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.700890341-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.704024774-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:20:34.726300073-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:21:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:28.504480327-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.664602829-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.688576608-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.692060753-04:00" level=debug msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.695751681-04:00" level=debug msg="Calling GET /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.699000934-04:00" level=debug msg="Calling GET /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.701273877-04:00" level=debug msg="Calling GET /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.704294369-04:00" level=debug msg="Calling GET /v1.26/info" Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us dockerd[10446]: time="2019-06-01T13:21:34.727622048-04:00" level=debug msg="failed to get networks for scope global: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: connect: connection refused\n" ___________________________________________________________________ # docker info Containers: 4 Running: 3 Paused: 0 Stopped: 1 Images: 6 Server Version: 18.09.6 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host kuryr macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84 runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30 init version: fec3683 Security Options: apparmor seccomp Profile: default Kernel Version: 4.15.0-45-generic Operating System: Ubuntu 18.04.1 LTS OSType: linux Architecture: x86_64 CPUs: 40 Total Memory: 251.8GiB Name: compute-1.zun1.labs1-pg0.clemson.cloudlab.us ID: Z2V3:UUZ2:GVDZ:PS3F:GFL5:NADG:HZDC:SPSD:LMNA:P2LW:FVIN:XHZS Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): true File Descriptors: 52 Goroutines: 62 System Time: 2019-06-01T13:15:35.719336887-04:00 EventsListeners: 0 Registry: https://index.docker.io/v1/ Labels: Experimental: false Cluster Store: etcd://192.168.0.1:2379 Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine WARNING: API is accessible on http://192.168.0.5:2375 without encryption. Access to the remote API is equivalent to root access on the host. Refer to the 'Docker daemon attack surface' section in the documentation for more information: https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface WARNING: No swap limit support <a href="https://bugs.launchpad.net/zun/+bug/1830450/comments/12/+download">Download full text</a> (58.9 KiB) <div class="comment-text editable-message-text" itemprop="commentText"><p>I couldn't find the previous error ( timeout waiting for exec session ready<br/> ) in the docker log. Could you re-run the Zun command and capture the log<br/> again?</p> <p>On Sat, Jun 1, 2019 at 1:35 PM fatis <email address hidden> wrote:</p> <p><span class="foldable-quoted">> # cat /etc/docker/<wbr/>daemon.<wbr/>json<br/> > "debug": true<br/> > # journalctl -u docker.service<br/> > # cat docker.log<br/> > -- Logs begin at Fri 2019-05-24 19:45:41 EDT, end at Sat 2019-06-01<br/> > 13:21:34 EDT. --<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us systemd[1]:<br/> > Starting Docker Application Container Engine...<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.858369071<wbr/>-04:00" level=info<br/> > msg="systemd-<wbr/>resolved is running, so using resolvconf:<br/> > /run/systemd/<wbr/>resolve/<wbr/>resolv.<wbr/>conf"<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903362201<wbr/>-04:00" level=info<br/> > msg="parsed scheme: \"unix\"" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903397841<wbr/>-04:00" level=info<br/> > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903502237<wbr/>-04:00" level=info<br/> > msg="parsed scheme: \"unix\"" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903518953<wbr/>-04:00" level=info<br/> > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903506998<wbr/>-04:00" level=info<br/> > msg="ccResolver<wbr/>Wrapper: sending new addresses to cc:<br/> > [{unix:<wbr/>///run/<wbr/>containerd/<wbr/>containerd.<wbr/>sock 0 <nil>}]" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903555369<wbr/>-04:00" level=info<br/> > msg="ClientConn switching balancer to \"pick_first\"" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903612640<wbr/>-04:00" level=info<br/> > msg="pickfirstB<wbr/>alancer: HandleSubConnSt<wbr/>ateChange: 0xc420822180, CONNECTING"<br/> > module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903684138<wbr/>-04:00" level=info<br/> > msg="ccResolver<wbr/>Wrapper: sending new addresses to cc:<br/> > [{unix:<wbr/>///run/<wbr/>containerd/<wbr/>containerd.<wbr/>sock 0 <nil>}]" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903743833<wbr/>-04:00" level=info<br/> > msg="ClientConn switching balancer to \"pick_first\"" module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903812509<wbr/>-04:00" level=info<br/> > msg="pickfirstB<wbr/>alancer: HandleSubConnSt<wbr/>ateChange: 0xc4207a2150, CONNECTING"<br/> > module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us<br/> > dockerd[8383]: time="2019-<wbr/>05-24T19:<wbr/>55:49.903835625<wbr/>-04:00" level=info<br/> > msg="pickfirstB<wbr/>alancer: HandleSubConnSt<wbr/>ateChange: 0xc420822180, READY"<br/> > module=grpc<br/> > May 24 19:55:49 compute-<wbr/>1.zun1.<wbr/>labs... <textarea style="width: 100%" rows="10">I couldn't find the previous error ( timeout waiting for exec session ready ) in the docker log. Could you re-run the Zun command and capture the log again? On Sat, Jun 1, 2019 at 1:35 PM fatis <1830450@bugs.launchpad.net> wrote: > # cat /etc/docker/daemon.json > "debug": true > # journalctl -u docker.service > # cat docker.log > -- Logs begin at Fri 2019-05-24 19:45:41 EDT, end at Sat 2019-06-01 > 13:21:34 EDT. -- > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Starting Docker Application Container Engine... > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.858369071-04:00" level=info > msg="systemd-resolved is running, so using resolvconf: > /run/systemd/resolve/resolv.conf" > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903362201-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903397841-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903502237-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903518953-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903506998-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903555369-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903612640-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420822180, CONNECTING" > module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903684138-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903743833-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903812509-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207a2150, CONNECTING" > module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903835625-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420822180, READY" > module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.903833860-04:00" level=info > msg="blockingPicker: the picked transport is not ready, loop back to > repick" module=grpc > May 24 19:55:49 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:49.904000665-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207a2150, READY" > module=grpc > May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:50.104124302-04:00" level=info > msg="Graph migration to content-addressability took 0.00 seconds" > May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:50.104474636-04:00" level=warning > msg="Your kernel does not support swap memory limit" > May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:50.104591093-04:00" level=warning > msg="Your kernel does not support cgroup rt period" > May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:50.104618391-04:00" level=warning > msg="Your kernel does not support cgroup rt runtime" > May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:50.105250169-04:00" level=info > msg="Loading containers: start." > May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:50.422816653-04:00" level=info > msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. > Daemon option --bip can be used to set a preferred IP address" > May 24 19:55:50 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:50.559727165-04:00" level=info > msg="Loading containers: done." > May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:51.208912044-04:00" level=info > msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 > May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:51.209074335-04:00" level=info > msg="Daemon has completed initialization" > May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:51.496957523-04:00" level=info > msg="API listen on /var/run/docker.sock" > May 24 19:55:51 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Started Docker Application Container Engine. > May 24 19:55:59 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T19:55:59.710302568-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Stopping Docker Application Container Engine... > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T20:01:30.789804236-04:00" level=info > msg="Processing signal 'terminated'" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[8383]: time="2019-05-24T20:01:30.790693612-04:00" level=info > msg="stopping event stream following graceful shutdown" error="<nil>" > module=libcontainerd namespace=moby > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Stopped Docker Application Container Engine. > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Starting Docker Application Container Engine... > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.861161946-04:00" level=info > msg="systemd-resolved is running, so using resolvconf: > /run/systemd/resolve/resolv.conf" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862590017-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862630186-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862683154-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862696670-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862760787-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862837705-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862779304-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862927302-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.862939509-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42070a200, CONNECTING" > module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.863017496-04:00" level=info > msg="blockingPicker: the picked transport is not ready, loop back to > repick" module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.863015360-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c7610, CONNECTING" > module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.863224369-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42070a200, READY" > module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.863254133-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c7610, READY" > module=grpc > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.864817636-04:00" level=info > msg="[graphdriver] using prior storage driver: overlay2" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.893527775-04:00" level=info > msg="Graph migration to content-addressability took 0.00 seconds" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.893819559-04:00" level=warning > msg="Your kernel does not support swap memory limit" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.893876995-04:00" level=warning > msg="Your kernel does not support cgroup rt period" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.893893904-04:00" level=warning > msg="Your kernel does not support cgroup rt runtime" > May 24 20:01:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:30.894433695-04:00" level=info > msg="Loading containers: start." > May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:31.133946971-04:00" level=info > msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. > Daemon option --bip can be used to set a preferred IP address" > May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:31.241225541-04:00" level=info > msg="Loading containers: done." > May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:31.471177743-04:00" level=info > msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 > May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:31.471267752-04:00" level=info > msg="Daemon has completed initialization" > May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:01:31.527275562-04:00" level=info > msg="API listen on /var/run/docker.sock" > May 24 20:01:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Started Docker Application Container Engine. > May 24 20:17:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > docker.service: Current command vanished from the unit file, execution of > the command list won't be resumed. > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Stopping Docker Application Container Engine... > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:17:37.489068376-04:00" level=info > msg="Processing signal 'terminated'" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:17:37.489933441-04:00" level=info > msg="stopping event stream following graceful shutdown" error="<nil>" > module=libcontainerd namespace=moby > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[11329]: time="2019-05-24T20:17:37.490408324-04:00" level=info > msg="stopping event stream following graceful shutdown" error="context > canceled" module=libcontainerd namespace=plugins.moby > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Stopped Docker Application Container Engine. > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Starting Docker Application Container Engine... > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.550025542-04:00" level=warning > msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU > DON'T KNOW WHAT YOU'RE DOING [!]" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.561837754-04:00" level=info > msg="systemd-resolved is running, so using resolvconf: > /run/systemd/resolve/resolv.conf" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.562840608-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.562874490-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.562931637-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.562945759-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563067876-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563072320-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563147871-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563102368-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563241106-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c6700, CONNECTING" > module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563300921-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42080a1a0, CONNECTING" > module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563476574-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42080a1a0, READY" > module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.563477762-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4200c6700, READY" > module=grpc > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.564952835-04:00" level=info > msg="[graphdriver] using prior storage driver: overlay2" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.602153582-04:00" level=info > msg="Graph migration to content-addressability took 0.00 seconds" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.602468861-04:00" level=warning > msg="Your kernel does not support swap memory limit" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.602531493-04:00" level=warning > msg="Your kernel does not support cgroup rt period" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.602562008-04:00" level=warning > msg="Your kernel does not support cgroup rt runtime" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.603144892-04:00" level=info > msg="Loading containers: start." > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.850881037-04:00" level=info > msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. > Daemon option --bip can be used to set a preferred IP address" > May 24 20:17:37 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:37.962900503-04:00" level=info > msg="Loading containers: done." > May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:38.210057261-04:00" level=info > msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 > May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:38.210152641-04:00" level=info > msg="Daemon has completed initialization" > May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:38.276678580-04:00" level=info > msg="API listen on /var/run/docker.sock" > May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-24T20:17:38.276721864-04:00" level=info > msg="API listen on 192.168.0.5:2375" > May 24 20:17:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Started Docker Application Container Engine. > May 25 11:56:43 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-25T11:56:43.488871946-04:00" level=warning > msg="Your kernel does not support swap limit capabilities,or the cgroup is > not mounted. Memory limited without swap." > May 25 12:01:42 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-25T12:01:42.586332098-04:00" level=info > msg="Container > 50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0 failed to > exit within 10 seconds of signal 15 - using the force" > May 25 12:01:42 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-25T12:01:42.783250781-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 25 13:07:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-25T13:07:12.249476332-04:00" level=error > msg="Not continuing with pull after error: errors:\ndenied: requested > access to the resource is denied\nunauthorized: authentication required\n" > May 25 13:07:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-25T13:07:12.249524106-04:00" level=info > msg="Ignoring extra error returned from registry: unauthorized: > authentication required" > May 28 09:28:01 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:28:01.044559035-04:00" level=error > msg="stream copy error: reading from a closed fifo" > May 28 09:28:01 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:28:01.044587325-04:00" level=error > msg="stream copy error: reading from a closed fifo" > May 28 09:28:01 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:28:01.115233194-04:00" level=error > msg="Error running exec > 9b266f33f7e6108d406268a63859eb68e144c39319c3c82f9a73c9cae8e8217c in > container: OCI runtime exec failed: exec failed: container_linux.go:345: > starting container process caused \"exec: \\\"/bin/bash\\\": stat > /bin/bash: no such file or directory\": unknown" > May 28 09:33:08 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:33:08.563751788-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 28 09:44:13 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:44:13.374623021-04:00" level=info > msg="Container > 50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0 failed to > exit within 10 seconds of signal 15 - using the force" > May 28 09:44:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:44:23.414223215-04:00" level=info > msg="Container 50fc977abf05 failed to exit within 10 seconds of kill - > trying direct SIGKILL" > May 28 09:47:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:47:26.078689928-04:00" level=info > msg="Container 50fc977abf05 failed to exit within 10 seconds of kill - > trying direct SIGKILL" > May 28 09:47:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:47:29.122526145-04:00" level=info > msg="Container > 50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0 failed to > exit within 3 seconds of signal 15 - using the force" > May 28 09:47:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:47:39.166464318-04:00" level=info > msg="Container 50fc977abf05 failed to exit within 10 seconds of kill - > trying direct SIGKILL" > May 28 09:53:43 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-28T09:53:43.256877588-04:00" level=warning > msg="Your kernel does not support swap limit capabilities,or the cgroup is > not mounted. Memory limited without swap." > May 29 10:32:57 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-29T10:32:57.052367435-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 29 10:33:08 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-29T10:33:08.319816062-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 29 10:33:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-29T10:33:12.932672654-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 29 12:48:13 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-29T12:48:13.998694211-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 29 13:15:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-29T13:15:26.696615580-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 30 09:05:45 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:05:45.901276733-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 30 09:09:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:09:12.955469478-04:00" level=error > msg="Not continuing with pull after error: errors:\ndenied: requested > access to the resource is denied\nunauthorized: authentication required\n" > May 30 09:09:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:09:12.955517276-04:00" level=info > msg="Ignoring extra error returned from registry: unauthorized: > authentication required" > May 30 09:12:53 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:12:53.009059583-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 30 09:36:12 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > docker.service: Current command vanished from the unit file, execution of > the command list won't be resumed. > May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Stopping Docker Application Container Engine... > May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:36:23.299783310-04:00" level=info > msg="Processing signal 'terminated'" > May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:36:23.510044573-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 30 09:36:23 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:36:23.886396108-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[12715]: time="2019-05-30T09:36:29.255297521-04:00" level=info > msg="stopping event stream following graceful shutdown" error="<nil>" > module=libcontainerd namespace=moby > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Stopped Docker Application Container Engine. > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Starting Docker Application Container Engine... > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.318196152-04:00" level=warning > msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU > DON'T KNOW WHAT YOU'RE DOING [!]" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.328938560-04:00" level=info > msg="systemd-resolved is running, so using resolvconf: > /run/systemd/resolve/resolv.conf" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.330861153-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.330967939-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.331099437-04:00" level=info > msg="parsed scheme: \"unix\"" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.331123399-04:00" level=info > msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.331471091-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.331610299-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.331537943-04:00" level=info > msg="ccResolverWrapper: sending new addresses to cc: > [{unix:///run/containerd/containerd.sock 0 <nil>}]" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.331788095-04:00" level=info > msg="ClientConn switching balancer to \"pick_first\"" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.332117669-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420ba4060, CONNECTING" > module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.332131455-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420b78180, CONNECTING" > module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.332146022-04:00" level=info > msg="blockingPicker: the picked transport is not ready, loop back to > repick" module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.332427173-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420ba4060, READY" > module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.332686847-04:00" level=info > msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420b78180, READY" > module=grpc > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.334234395-04:00" level=info > msg="[graphdriver] using prior storage driver: overlay2" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.369888614-04:00" level=info > msg="Graph migration to content-addressability took 0.00 seconds" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.370259821-04:00" level=warning > msg="Your kernel does not support swap memory limit" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.370342699-04:00" level=warning > msg="Your kernel does not support cgroup rt period" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.370376461-04:00" level=warning > msg="Your kernel does not support cgroup rt runtime" > May 30 09:36:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:29.371004766-04:00" level=info > msg="Loading containers: start." > May 30 09:36:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:30.057651750-04:00" level=info > msg="Removing stale sandbox > a463e6c4b0b64ff925c30ff12c76c7e29db2b332c26fdc7649214918ed7e4f28 > (50fc977abf0568d7eb04d655f66f567a1b54475a0aa08645e9fde37363e197a0)" > May 30 09:36:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:30.234660162-04:00" level=warning > msg="Error (Unable to complete atomic operation, key modified) deleting > object [endpoint > 63699a212dae47fecdc8078df2ac310513e27cd98531d157fe5b75fd538fd66e > fa7cbce00674806f4c3bf3c55c225d4c48b0670c1906e1c99d3ce9d540f826cf], > retrying...." > May 30 09:36:31 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:31.950540494-04:00" level=info > msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. > Daemon option --bip can be used to set a preferred IP address" > May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:32.091239729-04:00" level=info > msg="Loading containers: done." > May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:32.630717553-04:00" level=info > msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 > May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:32.630829877-04:00" level=info > msg="Daemon has completed initialization" > May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:32.707556989-04:00" level=info > msg="API listen on 192.168.0.5:2375" > May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-30T09:36:32.707584381-04:00" level=info > msg="API listen on /var/run/docker.sock" > May 30 09:36:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us systemd[1]: > Started Docker Application Container Engine. > May 31 10:45:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-31T10:45:38.594435640-04:00" level=info > msg="Container > 655b095307bdde41a75aad3ee654e71985fd6bad0e4c015ed9b656ff5d1cd920 failed to > exit within 10 seconds of signal 15 - using the force" > May 31 10:45:38 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-05-31T10:45:38.823583167-04:00" level=info > msg="ignoring event" module=libcontainerd namespace=moby > topic=/tasks/delete type="*events.TaskDelete" > Jun 01 13:13:54 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:13:54.770951837-04:00" level=info > msg="Got signal to reload configuration, reloading from: > /etc/docker/daemon.json" > Jun 01 13:13:54 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:13:54.771747636-04:00" level=info > msg="Reloaded configuration: > {\"mtu\":1500,\"pidfile\":\"/var/run/docker.pid\",\"data-root\":\"/var/lib/docker\",\"exec-root\":\"/var/run/docker\",\"group\":\"zun\",\"deprecated-key-path\":\"/etc/docker/key.json\",\"cluster-store\":\"etcd:// > 192.168.0.1:2379 > \",\"max-concurrent-downloads\":3,\"max-concurrent-uploads\":5,\"shutdown-timeout\":15,\"debug\":true,\"hosts\":[\"tcp:// > 192.168.0.5:2375 > \",\"unix:///var/run/docker.sock\"],\"log-level\":\"info\",\"swarm-default-advertise-addr\":\"\",\"swarm-raft-heartbeat-tick\":0,\"swarm-raft-election-tick\":0,\"metrics-addr\":\"\",\"log-driver\":\"json-file\",\"ip\":\"0.0.0.0\",\"icc\":true,\"iptables\":true,\"ip-forward\":true,\"ip-masq\":true,\"userland-proxy\":true,\"default-address-pools\":{},\"network-control-plane-mtu\":1500,\"disable-legacy-registry\":true,\"experimental\":false,\"containerd\":\"/run/containerd/containerd.sock\",\"builder\":{\"GC\":{}},\"runtimes\":{\"runc\":{\"path\":\"runc\"}},\"default-runtime\":\"runc\",\"oom-score-adjust\":-500,\"default-shm-size\":67108864,\"default-ipc-mode\":\"shareable\",\"resolv-conf\":\"/run/systemd/resolve/resolv.conf\"}" > Jun 01 13:14:00 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:00.631429844-04:00" level=debug > msg="Calling GET /_ping" > Jun 01 13:14:00 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:00.632011830-04:00" level=debug > msg="Calling GET /v1.39/info" > Jun 01 13:14:00 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:00.673263619-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:14:20 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:20.504062855-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.664901706-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.689117419-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.692574836-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.696289343-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.698938875-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.701271804-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.704400654-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:14:27 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:27.727719445-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:14:46 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:46.430200484-04:00" level=debug > msg="Calling GET /_ping" > Jun 01 13:14:46 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:14:46.431411964-04:00" level=debug > msg="Calling GET /v1.39/containers/json" > Jun 01 13:15:22 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:22.504416027-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.668542803-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.692771341-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.696259087-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.699904113-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.702294246-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.704802908-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.707913866-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:15:29 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:29.730612269-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:15:35 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:35.717889451-04:00" level=debug > msg="Calling GET /_ping" > Jun 01 13:15:35 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:35.718336063-04:00" level=debug > msg="Calling GET /v1.39/info" > Jun 01 13:15:35 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:15:35.760989788-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:16:24 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:24.503942833-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.664961401-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.689205294-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.692546135-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.696252604-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.698610508-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.701117700-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.704261463-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:16:30 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:16:30.726543208-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:17:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:26.504347290-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.664386055-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.688535243-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.691917116-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.695442614-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.697782366-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.700181957-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.703130620-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:17:32 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:17:32.724627595-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:18:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:26.531605753-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.142981125-04:00" level=debug > msg="Calling GET /_ping" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.143415870-04:00" level=debug > msg="Calling GET > /v1.39/containers/zun-9bd314af-a8e0-4aa0-96d7-eb69345d2e79/json" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.145724199-04:00" level=debug > msg="Calling POST > /v1.39/containers/zun-9bd314af-a8e0-4aa0-96d7-eb69345d2e79/exec" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.145855963-04:00" level=debug > msg="form data: > {\"AttachStderr\":true,\"AttachStdin\":true,\"AttachStdout\":true,\"Cmd\":[\"/bin/sh\"],\"Detach\":false,\"DetachKeys\":\"\",\"Env\":null,\"Privileged\":false,\"Tty\":false,\"User\":\"\",\"WorkingDir\":\"\"}" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.146333778-04:00" level=debug > msg="Calling POST > /v1.39/exec/e9c97eb44b73b258b165a01e6b218305e82c8a39187872fa26376abd75dff518/start" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.146407864-04:00" level=debug > msg="form data: {\"Detach\":false,\"Tty\":false}" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.146513845-04:00" level=debug > msg="starting exec command > e9c97eb44b73b258b165a01e6b218305e82c8a39187872fa26376abd75dff518 in > container cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.148885264-04:00" level=debug > msg="attach: stdout: begin" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.148910971-04:00" level=debug > msg="attach: stdin: begin" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.148940708-04:00" level=debug > msg="attach: stderr: begin" > Jun 01 13:18:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:28.300965501-04:00" level=debug > msg=event module=libcontainerd namespace=moby topic=/tasks/exec-started > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.664519878-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.688368815-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.691653911-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.695201603-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.697493585-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.699750823-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.702692702-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:18:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:33.724664601-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:39.848349267-04:00" level=debug > msg="Closing buffered stdin pipe" > Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:39.848415447-04:00" level=debug > msg="attach: stdin: end" > Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:39.935555994-04:00" level=debug > msg=event module=libcontainerd namespace=moby topic=/tasks/exit > Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:39.935648817-04:00" level=debug > msg="attach: stdout: end" > Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:39.935712539-04:00" level=debug > msg="attach: stderr: end" > Jun 01 13:18:39 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:18:39.935739458-04:00" level=debug > msg="attach done" > Jun 01 13:19:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:26.559287617-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.672470236-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.696243049-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.699493238-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.703072526-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.705361140-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.707579027-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.710513180-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:19:33 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:19:33.732635018-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:20:26 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:26.586968988-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.665040671-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.689059381-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.692514730-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.696117352-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.698505933-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.700890341-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.704024774-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:20:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:20:34.726300073-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:21:28 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:28.504480327-04:00" level=debug > msg="Calling GET > /v1.26/containers/json?all=1&limit=-1&filters=%7B%22name%22%3A+%5B%22zun-%22%5D%7D&trunc_cmd=0&size=0" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.664602829-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.688576608-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.692060753-04:00" level=debug > msg="Calling GET /v1.26/containers/json?all=0&limit=-1&trunc_cmd=0&size=0" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.695751681-04:00" level=debug > msg="Calling GET > /v1.26/containers/cee2dda204045194d4b01f3776247acd4873b4bbe0ba820b1e8bf329694a9e7e/json" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.699000934-04:00" level=debug > msg="Calling GET > /v1.26/containers/2be097c4e0785d207b44903ce1f43da513a7620ada57d82d891296d31b2ca367/json" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.701273877-04:00" level=debug > msg="Calling GET > /v1.26/containers/bad747f297b6a3a93de653e243dbfc6162bff74c32b2abed5a15f8d0578f64d9/json" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.704294369-04:00" level=debug > msg="Calling GET /v1.26/info" > Jun 01 13:21:34 compute-1.zun1.labs1-pg0.clemson.cloudlab.us > dockerd[10446]: time="2019-06-01T13:21:34.727622048-04:00" level=debug > msg="failed to get networks for scope global: client: etcd cluster is > unavailable or misconfigured; error #0: dial tcp 192.168.0.1:2379: > connect: connection refused\n" > ___________________________________________________________________ > # docker info > Containers: 4 > Running: 3 > Paused: 0 > Stopped: 1 > Images: 6 > Server Version: 18.09.6 > Storage Driver: overlay2 > Backing Filesystem: extfs > Supports d_type: true > Native Overlay Diff: true > Logging Driver: json-file > Cgroup Driver: cgroupfs > Plugins: > Volume: local > Network: bridge host kuryr macvlan null overlay > Log: awslogs fluentd gcplogs gelf journald json-file local logentries > splunk syslog > Swarm: inactive > Runtimes: runc > Default Runtime: runc > Init Binary: docker-init > containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84 > runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30 > init version: fec3683 > Security Options: > apparmor > seccomp > Profile: default > Kernel Version: 4.15.0-45-generic > Operating System: Ubuntu 18.04.1 LTS > OSType: linux > Architecture: x86_64 > CPUs: 40 > Total Memory: 251.8GiB > Name: compute-1.zun1.labs1-pg0.clemson.cloudlab.us > ID: Z2V3:UUZ2:GVDZ:PS3F:GFL5:NADG:HZDC:SPSD:LMNA:P2LW:FVIN:XHZS > Docker Root Dir: /var/lib/docker > Debug Mode (client): false > Debug Mode (server): true > File Descriptors: 52 > Goroutines: 62 > System Time: 2019-06-01T13:15:35.719336887-04:00 > EventsListeners: 0 > Registry: https://index.docker.io/v1/ > Labels: > Experimental: false > Cluster Store: etcd://192.168.0.1:2379 > Insecure Registries: > 127.0.0.0/8 > Live Restore Enabled: false > Product License: Community Engine > WARNING: API is accessible on http://192.168.0.5:2375 without encryption. > Access to the remote API is equivalent to root access on the > host. Refer > to the 'Docker daemon attack surface' section in the > documentation for > more information: > https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface > WARNING: No swap limit support > You received this bug notification because you are subscribed to Zun. > Matching subscriptions: zun > https://bugs.launchpad.net/bugs/1830450 > Title: > openstack appcontainer exec --interactive > Status in Zun: > Bug description: > I have this error when I exec intercatve container > # openstack appcontainer exec --interactive container /bin/sh > HTTPConnectionPool(host='130.127.133.20', port=2375): Max retries > exceeded with url: > /v1.35/exec/808ebed1841484c028d0f0684f7637fe28849592125607c55173155ca0f47668/start > (Caused by NewConnectionError('<urllib3.connection.HTTPConnection > object at 0x7f44ae9ef550>: Failed to establish a new connection: > [Errno 111] Connection refused',)) > /etc/systemd/system/docker.service.d/docker.conf #my compute node > [Service] > ExecStart= > ExecStart=/usr/bin/dockerd --group zun -H tcp://compute-1:2375 -H > unix:///var/run/docker.sock --cluster-store etcd://controller:2379 > To manage notifications about this bug go to: > https://bugs.launchpad.net/zun/+bug/1830450/+subscriptions <a href="https://bugs.launchpad.net/zun/+bug/1830450/comments/13/+download">Download full text</a> (45.1 KiB) <div class="comment-text editable-message-text" itemprop="commentText"><p>On Fri, May 31, 2019 at 9:20 PM fatis <email address hidden> wrote:</p> <p><span class="foldable-quoted">> Hi @hongbin,<br/> > I can't correct the problem, when I run on Compute Host:# docker exec<br/> > --interactive zun-ca694bec-<wbr/>2e89-4a3f-<wbr/>b78b-ad21c9fe93<wbr/>53 /bin/bash it work<br/> > perfectly<br/> > But on the controller # openstack --debug appcontainer exec --interactive<br/> > container /bin/sh it give Internal Server Error (HTTP 500)<br/> <p>If you run "docker exec ... /bin/bash" and it works, you might want to try<br/> "openstack appcontainer exec ... /bin/bash" (not /bin/sh). Just let me know<br/> if that works.</p> <p><span class="foldable-quoted">> _______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>_______<wbr/>____<br/> > #openstack --debug appcontainer exec --interactive container /bin/sh<br/> > WARNING: Failed to import plugin clustering.<br/> > START with options: --debug appcontainer exec --interactive container<br/> > /bin/sh<br/> > options: Namespace(<wbr/>access_<wbr/>key='', access_<wbr/>secret=<wbr/>'***', access_token='***',<br/> > access_<wbr/>token_endpoint=<wbr/>'', access_<wbr/>token_type=<wbr/>'', aodh_endpoint='',<br/> > application_<wbr/>credential_<wbr/>id='', application_<wbr/>credential_<wbr/>name=''<wbr/>,<br/> > application_<wbr/>credential_<wbr/>secret=<wbr/>'***', auth_type=<wbr/>'password'<wbr/>, auth_url='<br/> > <a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, cacert=None, cert='', client_id='',<br/> > client_<wbr/>secret=<wbr/>'***', cloud='', code='', consumer_key='',<br/> > consumer_<wbr/>secret=<wbr/>'***', debug=True, default_<wbr/>domain=<wbr/>'default'<wbr/>,<br/> > default_<wbr/>domain_<wbr/>id='', default_<wbr/>domain_<wbr/>name=''<wbr/>, deferred_<wbr/>help=False,<br/> > discovery_<wbr/>endpoint=<wbr/>'', domain_id='', domain_name='', endpoint='',<br/> > identity_<wbr/>provider=<wbr/>'', identity_<wbr/>provider_<wbr/>url='', insecure=None,<br/> > interface='public', key='', log_file=None, openid_scope='',<br/> > os_alarming_<wbr/>api_version=<wbr/>'2', os_beta_<wbr/>command=<wbr/>False,<br/> > os_compute_<wbr/>api_version=<wbr/>'', os_container_<wbr/>api_version=<wbr/>'1.20',<br/> > os_container_<wbr/>infra_api_<wbr/>version=<wbr/>'1', os_data_<wbr/>processing_<wbr/>api_version=<wbr/>'1.1',<br/> > os_data_<wbr/>processing_<wbr/>url='', os_database_<wbr/>api_version=<wbr/>'1',<br/> > os_dns_<wbr/>api_version=<wbr/>'2', os_identity_<wbr/>api_version=<wbr/>'3',<br/> > os_image_<wbr/>api_version=<wbr/>'2', os_key_<wbr/>manager_<wbr/>api_version=<wbr/>'1',<br/> > os_loadbalancer<wbr/>_api_version=<wbr/>'2.0', os_metrics_<wbr/>api_version=<wbr/>'1',<br/> > os_network_<wbr/>api_version=<wbr/>'', os_object_<wbr/>api_version=<wbr/>'',<br/> > os_orchestratio<wbr/>n_api_version=<wbr/>'1', os_project_id=None, os_project_<wbr/>name=None,<br/> > os_queues_<wbr/>api_version=<wbr/>'2', os_volume_<wbr/>api_version=<wbr/>'',<br/> > os_workflow_<wbr/>api_version=<wbr/>'2', passcode='', password='***', profile='',<br/> > project_<wbr/>domain_<wbr/>id='', project_<wbr/>domain_<wbr/>name='default'<wbr/>, project_id='',<br/> > project_<wbr/>name='admin'<wbr/>, protocol='', redirect_uri='', region_name='',<br/> > remote_<wbr/>project_<wbr/>domain_<wbr/>id='', remote_<wbr/>project_<wbr/>domain_<wbr/>name=''<wbr/>,<br/> > remote_<wbr/>project_<wbr/>id='', remote_<wbr/>project_<wbr/>name=''<wbr/>, roles='',<br/> > service_<wbr/>provider=<wbr/>'', service_<wbr/>provider_<wbr/>endpoint=<wbr/>'',<br/> > service_<wbr/>provider_<wbr/>entity_<wbr/>id='', system_scope='', timing=False, token='***',<br/> > trust_id='', url='', user='', user_domain_id='',<br/> > user_domain_<wbr/>name='default'<wbr/>, user_id='', username=<wbr/>'adminapi'<wbr/>,<br/> > verbose_level=3, verify=None)<br/> > Auth plugin password selected<br/> > auth_config_hook(): {'auth_type': 'password', 'beta_command': False,<br/> > u'image_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'key': None, 'database_<wbr/>api_version'<wbr/>:<br/> > '1', 'metrics_<wbr/>api_version'<wbr/>: '1', 'data_processin<wbr/>g_api_version'<wbr/>: '1.1',<br/> > 'auth_url': '<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, u'network_<wbr/>api_version'<wbr/>: u'2',<br/> > u'message': u'', u'image_format': u'qcow2', 'networks': [],<br/> > 'queues_<wbr/>api_version'<wbr/>: '... > I can't correct the problem, when I run on Compute Host:# docker exec > --interactive zun-ca694bec-2e89-4a3f-b78b-ad21c9fe9353 /bin/bash it work > perfectly > But on the controller # openstack --debug appcontainer exec --interactive > container /bin/sh it give Internal Server Error (HTTP 500) If you run "docker exec ... /bin/bash" and it works, you might want to try "openstack appcontainer exec ... /bin/bash" (not /bin/sh). Just let me know if that works. > ___________________________________________________________________ > #openstack --debug appcontainer exec --interactive container /bin/sh > WARNING: Failed to import plugin clustering. > START with options: --debug appcontainer exec --interactive container > /bin/sh > options: Namespace(access_key='', access_secret='***', access_token='***', > access_token_endpoint='', access_token_type='', aodh_endpoint='', > application_credential_id='', application_credential_name='', > application_credential_secret='***', auth_type='password', auth_url=' > http://controller:5000/v3', cacert=None, cert='', client_id='', > client_secret='***', cloud='', code='', consumer_key='', > consumer_secret='***', debug=True, default_domain='default', > default_domain_id='', default_domain_name='', deferred_help=False, > discovery_endpoint='', domain_id='', domain_name='', endpoint='', > identity_provider='', identity_provider_url='', insecure=None, > interface='public', key='', log_file=None, openid_scope='', > os_alarming_api_version='2', os_beta_command=False, > os_compute_api_version='', os_container_api_version='1.20', > os_container_infra_api_version='1', os_data_processing_api_version='1.1', > os_data_processing_url='', os_database_api_version='1', > os_dns_api_version='2', os_identity_api_version='3', > os_image_api_version='2', os_key_manager_api_version='1', > os_loadbalancer_api_version='2.0', os_metrics_api_version='1', > os_network_api_version='', os_object_api_version='', > os_orchestration_api_version='1', os_project_id=None, os_project_name=None, > os_queues_api_version='2', os_volume_api_version='', > os_workflow_api_version='2', passcode='', password='***', profile='', > project_domain_id='', project_domain_name='default', project_id='', > project_name='admin', protocol='', redirect_uri='', region_name='', > remote_project_domain_id='', remote_project_domain_name='', > remote_project_id='', remote_project_name='', roles='', > service_provider='', service_provider_endpoint='', > service_provider_entity_id='', system_scope='', timing=False, token='***', > trust_id='', url='', user='', user_domain_id='', > user_domain_name='default', user_id='', username='adminapi', > verbose_level=3, verify=None) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > defaults: {u'auth_type': 'password', u'status': u'active', > u'image_status_code_retries': 5, > u'baremetal_introspection_status_code_retries': 5, 'api_timeout': None, > 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': > u'neutron', 'key': None, u'interface': u'public', u'network_api_version': > u'2', u'message': u'', u'image_format': u'qcow2', > u'baremetal_status_code_retries': 5, 'verify': True, 'cert': None, > u'secgroup_source': u'neutron', u'object_store_api_version': u'1', > u'disable_vendor_agent': {}} > cloud cfg: {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'orchestration_api_version': '1', > 'database_api_version': '1', 'metrics_api_version': '1', > 'data_processing_api_version': '1.1', 'auth_url': ' > http://controller:5000/v3', u'network_api_version': u'2', u'message': > u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, > u'floating_ip_source': u'neutron', 'key': None, 'timing': False, > 'password': '***', 'cacert': None, 'key_manager_api_version': '1', > 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', > 'identity_api_version': '3', 'deferred_help': False, 'cert': None, > u'secgroup_source': u'neutron', u'status': u'active', > 'alarming_api_version': '2', 'container_api_version': '1.20', u'interface': > 'public', u'disable_vendor_agent': {}} > compute API version 2.1, cmd group openstack.compute.v2 > network API version 2, cmd group openstack.network.v2 > image API version 2, cmd group openstack.image.v2 > volume API version 2, cmd group openstack.volume.v2 > identity API version 3, cmd group openstack.identity.v3 > object_store API version 1, cmd group openstack.object_store.v1 > container API version 1.20, cmd group openstack.container.v1 > messaging API version 2, cmd group openstack.messaging.v2 > database API version 1, cmd group openstack.database.v1 > data_processing API version 1.1, cmd group openstack.data_processing.v1 > load_balancer API version 2.0, cmd group openstack.load_balancer.v2 > neutronclient API version 2, cmd group openstack.neutronclient.v2 > workflow_engine API version 2, cmd group openstack.workflow_engine.v2 > container_infra API version 1, cmd group openstack.container_infra.v1 > orchestration API version 1, cmd group openstack.orchestration.v1 > dns API version 2, cmd group openstack.dns.v2 > key_manager API version 1, cmd group openstack.key_manager.v1 > metric API version 1, cmd group openstack.metric.v1 > alarming API version 2, cmd group openstack.alarming.v2 > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > command: appcontainer exec -> zunclient.osc.v1.containers.ExecContainer > (auth=True) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', 'additional_user_agent': > [('osc-lib', '1.12.1')], u'network_api_version': u'2', u'message': u'', > u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Using auth plugin: password > Using parameters {'username': 'adminapi', 'project_name': 'admin', > 'user_domain_name': 'default', 'auth_url': 'http://controller:5000/v3', > 'password': '***', 'project_domain_name': 'default'} > Get auth_ref > REQ: curl -g -i -X GET http://controller:5000/v3 -H "Accept: > application/json" -H "User-Agent: openstacksdk/0.28.0 keystoneauth1/3.14.0 > python-requests/2.22.0 CPython/2.7.15rc1" > Starting new HTTP connection (1): controller:5000 > http://controller:5000 "GET /v3 HTTP/1.1" 200 250 > RESP: [200] Connection: Keep-Alive Content-Length: 250 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:11 GMT Keep-Alive: timeout=5, > max=100 Server: Apache/2.4.29 (Ubuntu) Vary: X-Auth-Token > x-openstack-request-id: req-87b586e0-81e9-4adf-a6fb-cf7260f12490 > RESP BODY: {"version": {"status": "stable", "updated": > "2018-10-15T00:00:00Z", "media-types": [{"base": "application/json", > "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.11", > "links": [{"href": "http://controller:5000/v3/", "rel": "self"}]}} > GET call to http://controller:5000/v3 used request id > req-87b586e0-81e9-4adf-a6fb-cf7260f12490 > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["P3f93tEbQgqYfj-DWeDt0g"], "issued_at": "2019-06-01T01:08:12.000000Z"}} > run(Namespace(command=[u'/bin/sh'], container=u'container', > interactive=True)) > take_action(Namespace(command=[u'/bin/sh'], container=u'container', > interactive=True)) > Instantiating zun client: <class 'zunclient.v1.client.Client'> > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-01T05:08:12.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["iGpHpBe5T-2Wl_gJML5UEA"], "issued_at": "2019-06-01T01:08:12.000000Z"}} > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" > Starting new HTTP connection (1): controller:9517 > http://controller:9517 "POST > /v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > HTTP/1.1" 200 310 > RESP: [200] Connection: keep-alive Content-Length: 310 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:13 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-060272d5-6505-4352-9e78-7e5e2761947f > RESP BODY: {"output": null, "exec_id": > "e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", > "proxy_url": > "ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9", > "exit_code": null} > POST call to container for > http://controller:9517/v1/containers/container/execute?command=%22%2Fbin%2Fsh%22&run=False&interactive=True > used request id req-060272d5-6505-4352-9e78-7e5e2761947f > connecting to: > ws://controller:6784/?token=4a47528d-3526-40d0-8114-45046cf6ffbf&uuid=490b8830-414d-42f5-a448-cb119b1482b4&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9 > connected to container, press Enter to continue > type ~. to disconnect > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6bac51e24481f98ebc8b2071971b0cfce36ddbc4e394a204aad14106bd308da3" > http://controller:9517 "POST > /v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > HTTP/1.1" 500 261 > RESP: [500] Connection: keep-alive Content-Length: 261 Content-Type: > application/json Date: Sat, 01 Jun 2019 01:08:23 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-af129bd8-36af-4dbc-9446-54d181d5293b > RESP BODY: {"errors": [{"status": 500, "code": "container.", "links": [], > "title": "Internal Server Error", "detail": "Your request could not be > handled because of a problem in the server. Error Correlation id is: > 0c19d0a7-339b-47b8-b37a-d8db930a1d01", "request_id": ""}]} > POST call to container for > http://controller:9517/v1/containers/container/execute_resize?h=36&exec_id=e1ddb970cf09ac9f5109da6f0461f7661178a51e7f387b1b121d79e79dbb59b9&w=142 > used request id req-af129bd8-36af-4dbc-9446-54d181d5293b > Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > clean_up ExecContainer: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 136, in run > ret_val = super(OpenStackShell, self).run(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 281, in > result = self.run_subcommand(remainder) > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 176, in run_subcommand > ret_value = super(OpenStackShell, self).run_subcommand(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-af129bd8-36af-4dbc-9446-54d181d5293b) > END return value: 1 > You received this bug notification because you are subscribed to Zun. > Matching subscriptions: zun > https://bugs.launchpad.net/bugs/1830450 > Title: > openstack appcontainer exec --interactive > Status in Zun: > Bug description: > I have this error when I exec intercatve container > # openstack appcontainer exec --interactive container /bin/sh > HTTPConnectionPool(host='130.127.133.20', port=2375): Max retries > exceeded with url: > /v1.35/exec/808ebed1841484c028d0f0684f7637fe28849592125607c55173155ca0f47668/start > (Caused by NewConnectionError('<urllib3.connection.HTTPConnection > object at 0x7f44ae9ef550>: Failed to establish a new connection: > [Errno 111] Connection refused',)) > /etc/systemd/system/docker.service.d/docker.conf #my compute node > [Service] > ExecStart= > ExecStart=/usr/bin/dockerd --group zun -H tcp://compute-1:2375 -H > unix:///var/run/docker.sock --cluster-store etcd://controller:2379 > To manage notifications about this bug go to: > https://bugs.launchpad.net/zun/+bug/1830450/+subscriptions <div class="comment-text editable-message-text" itemprop="commentText"><p>No, it isn't</p> <p># openstack --debug appcontainer exec --interactive container2 /bin/bash<br/> WARNING: Failed to import plugin clustering.<br/> START with options: --debug appcontainer exec --interactive container2 /bin/bash<br/> options: Namespace(<wbr/>access_<wbr/>key='', access_<wbr/>secret=<wbr/>'***', access_token='***', access_<wbr/>token_endpoint=<wbr/>'', access_<wbr/>token_type=<wbr/>'', aodh_endpoint='', application_<wbr/>credential_<wbr/>id='', application_<wbr/>credential_<wbr/>name=''<wbr/>, application_<wbr/>credential_<wbr/>secret=<wbr/>'***', auth_type=<wbr/>'password'<wbr/>, auth_url='<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, cacert=None, cert='', client_id='', client_<wbr/>secret=<wbr/>'***', cloud='', code='', consumer_key='', consumer_<wbr/>secret=<wbr/>'***', debug=True, default_<wbr/>domain=<wbr/>'default'<wbr/>, default_<wbr/>domain_<wbr/>id='', default_<wbr/>domain_<wbr/>name=''<wbr/>, deferred_<wbr/>help=False, discovery_<wbr/>endpoint=<wbr/>'', domain_id='', domain_name='', endpoint='', identity_<wbr/>provider=<wbr/>'', identity_<wbr/>provider_<wbr/>url='', insecure=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_<wbr/>api_version=<wbr/>'2', os_beta_<wbr/>command=<wbr/>False, os_compute_<wbr/>api_version=<wbr/>'', os_container_<wbr/>api_version=<wbr/>'1.20', os_container_<wbr/>infra_api_<wbr/>version=<wbr/>'1', os_data_<wbr/>processing_<wbr/>api_version=<wbr/>'1.1', os_data_<wbr/>processing_<wbr/>url='', os_database_<wbr/>api_version=<wbr/>'1', os_dns_<wbr/>api_version=<wbr/>'2', os_identity_<wbr/>api_version=<wbr/>'3', os_image_<wbr/>api_version=<wbr/>'2', os_key_<wbr/>manager_<wbr/>api_version=<wbr/>'1', os_loadbalancer<wbr/>_api_version=<wbr/>'2.0', os_metrics_<wbr/>api_version=<wbr/>'1', os_network_<wbr/>api_version=<wbr/>'', os_object_<wbr/>api_version=<wbr/>'', os_orchestratio<wbr/>n_api_version=<wbr/>'1', os_project_id=None, os_project_<wbr/>name=None, os_queues_<wbr/>api_version=<wbr/>'2', os_volume_<wbr/>api_version=<wbr/>'', os_workflow_<wbr/>api_version=<wbr/>'2', passcode='', password='***', profile='', project_<wbr/>domain_<wbr/>id='', project_<wbr/>domain_<wbr/>name='default'<wbr/>, project_id='', project_<wbr/>name='admin'<wbr/>, protocol='', redirect_uri='', region_name='', remote_<wbr/>project_<wbr/>domain_<wbr/>id='', remote_<wbr/>project_<wbr/>domain_<wbr/>name=''<wbr/>, remote_<wbr/>project_<wbr/>id='', remote_<wbr/>project_<wbr/>name=''<wbr/>, roles='', service_<wbr/>provider=<wbr/>'', service_<wbr/>provider_<wbr/>endpoint=<wbr/>'', service_<wbr/>provider_<wbr/>entity_<wbr/>id='', system_scope='', timing=False, token='***', trust_id='', url='', user='', user_domain_id='', user_domain_<wbr/>name='default'<wbr/>, user_id='', username=<wbr/>'adminapi'<wbr/>, verbose_level=3, verify=None)<br/> Auth plugin password selected<br/> auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'key': None, 'database_<wbr/>api_version'<wbr/>: '1', 'metrics_<wbr/>api_version'<wbr/>: '1', 'data_processin<wbr/>g_api_version'<wbr/>: '1.1', 'auth_url': '<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, u'network_<wbr/>api_version'<wbr/>: u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_<wbr/>api_version'<wbr/>: '2', 'verify': True, 'dns_api_version': '2', u'object_<wbr/>store_api_<wbr/>version'<wbr/>: u'1', 'username': 'adminapi', 'container_<wbr/>infra_api_<wbr/>version'<wbr/>: '1', 'loadbalancer_<wbr/>api_version'<wbr/>: '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_<wbr/>introspection_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'api_timeout': None, 'image_<wbr/>api_version'<wbr/>: '2', 'auth': {'user_<wbr/>domain_<wbr/>name': 'default', 'project_name': 'admin', 'project_<wbr/>domain_<wbr/>name': 'default'}, 'default_domain': 'default', 'container_<wbr/>api_version'<wbr/>: '1.20', u'image_<wbr/>api_use_<wbr/>tasks': False, u'floating_<wbr/>ip_source'<wbr/>: u'neutron', 'orchestration_<wbr/>api_version'<wbr/>: '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_<wbr/>api_version'<wbr/>: '1', 'workflow_<wbr/>api_version'<wbr/>: '2', u'barem...</p></div> # openstack --debug appcontainer exec --interactive container2 /bin/bash WARNING: Failed to import plugin clustering. START with options: --debug appcontainer exec --interactive container2 /bin/bash options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', aodh_endpoint='', application_credential_id='', application_credential_name='', application_credential_secret='***', auth_type='password', auth_url='http://controller:5000/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_api_version='2', os_beta_command=False, os_compute_api_version='', os_container_api_version='1.20', os_container_infra_api_version='1', os_data_processing_api_version='1.1', os_data_processing_url='', os_database_api_version='1', os_dns_api_version='2', os_identity_api_version='3', os_image_api_version='2', os_key_manager_api_version='1', os_loadbalancer_api_version='2.0', os_metrics_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_queues_api_version='2', os_volume_api_version='', os_workflow_api_version='2', passcode='', password='***', profile='', project_domain_id='', project_domain_name='default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='', remote_project_domain_id='', remote_project_domain_name='', remote_project_id='', remote_project_name='', roles='', service_provider='', service_provider_endpoint='', service_provider_entity_id='', system_scope='', timing=False, token='***', trust_id='', url='', user='', user_domain_id='', user_domain_name='default', user_id='', username='adminapi', verbose_level=3, verify=None) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} defaults: {u'auth_type': 'password', u'status': u'active', u'image_status_code_retries': 5, u'baremetal_introspection_status_code_retries': 5, 'api_timeout': None, 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, u'interface': u'public', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'baremetal_status_code_retries': 5, 'verify': True, 'cert': None, u'secgroup_source': u'neutron', u'object_store_api_version': u'1', u'disable_vendor_agent': {}} cloud cfg: {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'orchestration_api_version': '1', 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'container_api_version': '1.20', u'interface': 'public', u'disable_vendor_agent': {}} compute API version 2.1, cmd group openstack.compute.v2 network API version 2, cmd group openstack.network.v2 image API version 2, cmd group openstack.image.v2 volume API version 2, cmd group openstack.volume.v2 identity API version 3, cmd group openstack.identity.v3 object_store API version 1, cmd group openstack.object_store.v1 container API version 1.20, cmd group openstack.container.v1 messaging API version 2, cmd group openstack.messaging.v2 database API version 1, cmd group openstack.database.v1 data_processing API version 1.1, cmd group openstack.data_processing.v1 load_balancer API version 2.0, cmd group openstack.load_balancer.v2 neutronclient API version 2, cmd group openstack.neutronclient.v2 workflow_engine API version 2, cmd group openstack.workflow_engine.v2 container_infra API version 1, cmd group openstack.container_infra.v1 orchestration API version 1, cmd group openstack.orchestration.v1 dns API version 2, cmd group openstack.dns.v2 key_manager API version 1, cmd group openstack.key_manager.v1 metric API version 1, cmd group openstack.metric.v1 alarming API version 2, cmd group openstack.alarming.v2 Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} command: appcontainer exec -> zunclient.osc.v1.containers.ExecContainer (auth=True) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'image_status_code_retries': '5', 'key': None, 'database_api_version': '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:5000/v3', 'additional_user_agent': [('osc-lib', '1.12.1')], u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'adminapi', 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, 'image_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'container_api_version': '1.20', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'orchestration_api_version': '1', 'timing': False, 'password': '***', 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': 'public', u'disable_vendor_agent': {}} Using auth plugin: password Using parameters {'username': 'adminapi', 'project_name': 'admin', 'user_domain_name': 'default', 'auth_url': 'http://controller:5000/v3', 'password': '***', 'project_domain_name': 'default'} Get auth_ref REQ: curl -g -i -X GET http://controller:5000/v3 -H "Accept: application/json" -H "User-Agent: openstacksdk/0.28.0 keystoneauth1/3.14.0 python-requests/2.22.0 CPython/2.7.15rc1" Starting new HTTP connection (1): controller:5000 http://controller:5000 "GET /v3 HTTP/1.1" 200 250 RESP: [200] Connection: Keep-Alive Content-Length: 250 Content-Type: application/json Date: Sun, 02 Jun 2019 00:35:55 GMT Keep-Alive: timeout=5, max=100 Server: Apache/2.4.29 (Ubuntu) Vary: X-Auth-Token x-openstack-request-id: req-943ebe3a-aef8-4b96-8874-2727410e31c7 RESP BODY: {"version": {"status": "stable", "updated": "2018-10-15T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.11", "links": [{"href": "http://controller:5000/v3/", "rel": "self"}]}} GET call to http://controller:5000/v3 used request id req-943ebe3a-aef8-4b96-8874-2727410e31c7 Making authentication request to http://controller:5000/v3/auth/tokens http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": "2019-06-02T04:35:56.000000Z", "project": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a128816d407542ee836bd30d8c36484d"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": [{"url": "http://controller:8774/v2.1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": "http://controller:8774/v2.1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3b4eef8ae0d64c1396386a5557d50560"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": [{"url": "http://controller:9001/v2", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": "http://controller:9001/v2", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": [{"url": "http://controller:9292", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": [{"url": "http://controller:5000/v3", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": "http://controller:5000/v3", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": "keystone"}, {"endpoints": [{"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3827d192568f4b1e8b1901bdedbe1137"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "916be6147a2d4df293ebf375abd6fed2"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "56aebd20ee19475cae24f2e558de4b2d"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": [{"url": "http://controller:9511/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": "http://controller:9511/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", "name": "magnum"}, {"endpoints": [{"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "52504619399d467ba64a1f232129768c"}, {"url": "http://controller:8080/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": [{"url": "http://controller:8041", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": [{"url": "http://controller:8778", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": "ceilometer"}, {"endpoints": [{"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0145b180de764baabb3f8fc15e5e0883"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b6006479263b4faf81963f21f523e11d"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": [{"url": "http://controller:8000/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "7b4e560125674762b7d516605986f5aa"}, {"url": "http://controller:8000/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "04d221684116407981f9d00d177b077a"}, {"url": "http://controller:9517/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "848b8262069a4c4c91716c52644783d8"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "password_expires_at": null, "name": "adminapi", "id": "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": ["2yQsgqMjRMaIPXiDC0VwlA"], "issued_at": "2019-06-02T00:35:56.000000Z"}} run(Namespace(command=[u'/bin/bash'], container=u'container2', interactive=True)) take_action(Namespace(command=[u'/bin/bash'], container=u'container2', interactive=True)) Instantiating zun client: <class 'zunclient.v1.client.Client'> Making authentication request to http://controller:5000/v3/auth/tokens http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": "2019-06-02T04:35:56.000000Z", "project": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a128816d407542ee836bd30d8c36484d"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": "http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": [{"url": "http://controller:8774/v2.1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": "http://controller:8774/v2.1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3b4eef8ae0d64c1396386a5557d50560"}, {"url": "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": [{"url": "http://controller:9001/v2", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": "http://controller:9001/v2", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": [{"url": "http://controller:9292", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": [{"url": "http://controller:5000/v3", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": "http://controller:5000/v3", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": "keystone"}, {"endpoints": [{"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "3827d192568f4b1e8b1901bdedbe1137"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "916be6147a2d4df293ebf375abd6fed2"}, {"url": "http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "56aebd20ee19475cae24f2e558de4b2d"}, {"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": [{"url": "http://controller:9511/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": "http://controller:9511/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", "name": "magnum"}, {"endpoints": [{"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "52504619399d467ba64a1f232129768c"}, {"url": "http://controller:8080/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": [{"url": "http://controller:8041", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": [{"url": "http://controller:8778", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": "ceilometer"}, {"endpoints": [{"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "0145b180de764baabb3f8fc15e5e0883"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b6006479263b4faf81963f21f523e11d"}, {"url": "http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": [{"url": "http://controller:8000/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "7b4e560125674762b7d516605986f5aa"}, {"url": "http://controller:8000/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "04d221684116407981f9d00d177b077a"}, {"url": "http://controller:9517/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "848b8262069a4c4c91716c52644783d8"}, {"url": "http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "password_expires_at": null, "name": "adminapi", "id": "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": ["vrRr0wvdRnOG0uM2hl19_A"], "issued_at": "2019-06-02T00:35:56.000000Z"}} REQ: curl -g -i -X POST http://controller:9517/v1/containers/container2/execute?command=%22%2Fbin%2Fbash%22&run=False&interactive=True -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: application/json" -H "OpenStack-API-Version: container 1.20" -H "User-Agent: None" -H "X-Auth-Token: {SHA256}6cd9644ef619ef1d55102a08eabcacc7e649405e31ceb7102219cf77e6951a50" Starting new HTTP connection (1): controller:9517 http://controller:9517 "POST /v1/containers/container2/execute?command=%22%2Fbin%2Fbash%22&run=False&interactive=True HTTP/1.1" 200 310 RESP: [200] Connection: keep-alive Content-Length: 310 Content-Type: application/json Date: Sun, 02 Jun 2019 00:35:57 GMT Openstack-Api-Maximum-Version: container 1.22 Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: req-fc701b80-a45f-4216-8790-c32c58190273 RESP BODY: {"output": null, "exec_id": "b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa", "proxy_url": "ws://controller:6784/?token=bd15f3a6-feb1-47f9-befe-6793a6c017eb&uuid=93acec9f-0234-44d9-a796-6e06e31fc90c&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa", "exit_code": null} POST call to container for http://controller:9517/v1/containers/container2/execute?command=%22%2Fbin%2Fbash%22&run=False&interactive=True used request id req-fc701b80-a45f-4216-8790-c32c58190273 connecting to: ws://controller:6784/?token=bd15f3a6-feb1-47f9-befe-6793a6c017eb&uuid=93acec9f-0234-44d9-a796-6e06e31fc90c&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa connected to container2, press Enter to continue type ~. to disconnect REQ: curl -g -i -X POST http://controller:9517/v1/containers/container2/execute_resize?h=36&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa&w=142 -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: application/json" -H "OpenStack-API-Version: container 1.20" -H "User-Agent: None" -H "X-Auth-Token: {SHA256}6cd9644ef619ef1d55102a08eabcacc7e649405e31ceb7102219cf77e6951a50" http://controller:9517 "POST /v1/containers/container2/execute_resize?h=36&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa&w=142 HTTP/1.1" 500 261 RESP: [500] Connection: keep-alive Content-Length: 261 Content-Type: application/json Date: Sun, 02 Jun 2019 00:36:07 GMT Openstack-Api-Maximum-Version: container 1.22 Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: req-d8514136-a496-417b-b202-e4b63ee49d9b RESP BODY: {"errors": [{"status": 500, "code": "container.", "links": [], "title": "Internal Server Error", "detail": "Your request could not be handled because of a problem in the server. Error Correlation id is: e873667c-cfc6-495a-97c2-86411845836d", "request_id": ""}]} POST call to container for http://controller:9517/v1/containers/container2/execute_resize?h=36&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa&w=142 used request id req-d8514136-a496-417b-b202-e4b63ee49d9b Internal Server Error (HTTP 500) (Request-ID: req-d8514136-a496-417b-b202-e4b63ee49d9b) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", line 537, in take_action websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 379, in do_exec wscls.handle_resize() File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 196, in handle_resize self.tty_resize(height=rows, width=cols) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 294, in tty_resize self.cs.containers.execute_resize(self.id, self.exec_id, width, height) File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 165, in execute_resize qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 128, in _action **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 367, in json_request resp = self._http_request(url, method, **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 350, in _http_request error_json.get('debuginfo'), method, url) InternalServerError: Internal Server Error (HTTP 500) (Request-ID: req-d8514136-a496-417b-b202-e4b63ee49d9b) clean_up ExecContainer: Internal Server Error (HTTP 500) (Request-ID: req-d8514136-a496-417b-b202-e4b63ee49d9b) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 136, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 281, in run result = self.run_subcommand(remainder) File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 176, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", line 537, in take_action websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 379, in do_exec wscls.handle_resize() File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 196, in handle_resize self.tty_resize(height=rows, width=cols) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", line 294, in tty_resize self.cs.containers.execute_resize(self.id, self.exec_id, width, height) File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 165, in execute_resize qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] File "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line 128, in _action **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 367, in json_request resp = self._http_request(url, method, **kwargs) File "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", line 350, in _http_request error_json.get('debuginfo'), method, url) InternalServerError: Internal Server Error (HTTP 500) (Request-ID: req-d8514136-a496-417b-b202-e4b63ee49d9b) END return value: 1 <div class="comment-text editable-message-text" itemprop="commentText"><p>Could you paste me your Docker Daemon log again?</p> <p>On Sat, Jun 1, 2019 at 8:45 PM fatis <email address hidden> wrote:</p> <p><span class="foldable-quoted">> No, it isn't<br/> > # openstack --debug appcontainer exec --interactive container2 /bin/bash<br/> > WARNING: Failed to import plugin clustering.<br/> > START with options: --debug appcontainer exec --interactive container2<br/> > /bin/bash<br/> > options: Namespace(<wbr/>access_<wbr/>key='', access_<wbr/>secret=<wbr/>'***', access_token='***',<br/> > access_<wbr/>token_endpoint=<wbr/>'', access_<wbr/>token_type=<wbr/>'', aodh_endpoint='',<br/> > application_<wbr/>credential_<wbr/>id='', application_<wbr/>credential_<wbr/>name=''<wbr/>,<br/> > application_<wbr/>credential_<wbr/>secret=<wbr/>'***', auth_type=<wbr/>'password'<wbr/>, auth_url='<br/> > <a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, cacert=None, cert='', client_id='',<br/> > client_<wbr/>secret=<wbr/>'***', cloud='', code='', consumer_key='',<br/> > consumer_<wbr/>secret=<wbr/>'***', debug=True, default_<wbr/>domain=<wbr/>'default'<wbr/>,<br/> > default_<wbr/>domain_<wbr/>id='', default_<wbr/>domain_<wbr/>name=''<wbr/>, deferred_<wbr/>help=False,<br/> > discovery_<wbr/>endpoint=<wbr/>'', domain_id='', domain_name='', endpoint='',<br/> > identity_<wbr/>provider=<wbr/>'', identity_<wbr/>provider_<wbr/>url='', insecure=None,<br/> > interface='public', key='', log_file=None, openid_scope='',<br/> > os_alarming_<wbr/>api_version=<wbr/>'2', os_beta_<wbr/>command=<wbr/>False,<br/> > os_compute_<wbr/>api_version=<wbr/>'', os_container_<wbr/>api_version=<wbr/>'1.20',<br/> > os_container_<wbr/>infra_api_<wbr/>version=<wbr/>'1', os_data_<wbr/>processing_<wbr/>api_version=<wbr/>'1.1',<br/> > os_data_<wbr/>processing_<wbr/>url='', os_database_<wbr/>api_version=<wbr/>'1',<br/> > os_dns_<wbr/>api_version=<wbr/>'2', os_identity_<wbr/>api_version=<wbr/>'3',<br/> > os_image_<wbr/>api_version=<wbr/>'2', os_key_<wbr/>manager_<wbr/>api_version=<wbr/>'1',<br/> > os_loadbalancer<wbr/>_api_version=<wbr/>'2.0', os_metrics_<wbr/>api_version=<wbr/>'1',<br/> > os_network_<wbr/>api_version=<wbr/>'', os_object_<wbr/>api_version=<wbr/>'',<br/> > os_orchestratio<wbr/>n_api_version=<wbr/>'1', os_project_id=None, os_project_<wbr/>name=None,<br/> > os_queues_<wbr/>api_version=<wbr/>'2', os_volume_<wbr/>api_version=<wbr/>'',<br/> > os_workflow_<wbr/>api_version=<wbr/>'2', passcode='', password='***', profile='',<br/> > project_<wbr/>domain_<wbr/>id='', project_<wbr/>domain_<wbr/>name='default'<wbr/>, project_id='',<br/> > project_<wbr/>name='admin'<wbr/>, protocol='', redirect_uri='', region_name='',<br/> > remote_<wbr/>project_<wbr/>domain_<wbr/>id='', remote_<wbr/>project_<wbr/>domain_<wbr/>name=''<wbr/>,<br/> > remote_<wbr/>project_<wbr/>id='', remote_<wbr/>project_<wbr/>name=''<wbr/>, roles='',<br/> > service_<wbr/>provider=<wbr/>'', service_<wbr/>provider_<wbr/>endpoint=<wbr/>'',<br/> > service_<wbr/>provider_<wbr/>entity_<wbr/>id='', system_scope='', timing=False, token='***',<br/> > trust_id='', url='', user='', user_domain_id='',<br/> > user_domain_<wbr/>name='default'<wbr/>, user_id='', username=<wbr/>'adminapi'<wbr/>,<br/> > verbose_level=3, verify=None)<br/> > Auth plugin password selected<br/> > auth_config_hook(): {'auth_type': 'password', 'beta_command': False,<br/> > u'image_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'key': None, 'database_<wbr/>api_version'<wbr/>:<br/> > '1', 'metrics_<wbr/>api_version'<wbr/>: '1', 'data_processin<wbr/>g_api_version'<wbr/>: '1.1',<br/> > 'auth_url': '<a rel="nofollow" href="http://controller:5000/v3'">http://<wbr/>controller:<wbr/>5000/v3'</a>, u'network_<wbr/>api_version'<wbr/>: u'2',<br/> > u'message': u'', u'image_format': u'qcow2', 'networks': [],<br/> > 'queues_<wbr/>api_version'<wbr/>: '2', 'verify': True, 'dns_api_version': '2',<br/> > u'object_<wbr/>store_api_<wbr/>version'<wbr/>: u'1', 'username': 'adminapi',<br/> > 'container_<wbr/>infra_api_<wbr/>version'<wbr/>: '1', 'loadbalancer_<wbr/>api_version'<wbr/>: '2.0',<br/> > 'verbose_level': 3, 'region_name': '',<br/> > u'baremetal_<wbr/>introspection_<wbr/>status_<wbr/>code_retries'<wbr/>: '5', 'api_timeout': None,<br/> > 'image_<wbr/>api_version'<wbr/>: '2', 'auth': {'user_<wbr/>domain_<wbr/>name': 'default',<br/> > 'project_name': 'admin', 'project_<wbr/>domain_<wbr/>name': 'default'},<br/> > 'default_domain': 'default', 'container_<wbr/>api_version'<wbr/>: '1... <textarea style="width: 100%" rows="10">Could you paste me your Docker Daemon log again? On Sat, Jun 1, 2019 at 8:45 PM fatis <1830450@bugs.launchpad.net> wrote: > No, it isn't > # openstack --debug appcontainer exec --interactive container2 /bin/bash > WARNING: Failed to import plugin clustering. > START with options: --debug appcontainer exec --interactive container2 > /bin/bash > options: Namespace(access_key='', access_secret='***', access_token='***', > access_token_endpoint='', access_token_type='', aodh_endpoint='', > application_credential_id='', application_credential_name='', > application_credential_secret='***', auth_type='password', auth_url=' > http://controller:5000/v3', cacert=None, cert='', client_id='', > client_secret='***', cloud='', code='', consumer_key='', > consumer_secret='***', debug=True, default_domain='default', > default_domain_id='', default_domain_name='', deferred_help=False, > discovery_endpoint='', domain_id='', domain_name='', endpoint='', > identity_provider='', identity_provider_url='', insecure=None, > interface='public', key='', log_file=None, openid_scope='', > os_alarming_api_version='2', os_beta_command=False, > os_compute_api_version='', os_container_api_version='1.20', > os_container_infra_api_version='1', os_data_processing_api_version='1.1', > os_data_processing_url='', os_database_api_version='1', > os_dns_api_version='2', os_identity_api_version='3', > os_image_api_version='2', os_key_manager_api_version='1', > os_loadbalancer_api_version='2.0', os_metrics_api_version='1', > os_network_api_version='', os_object_api_version='', > os_orchestration_api_version='1', os_project_id=None, os_project_name=None, > os_queues_api_version='2', os_volume_api_version='', > os_workflow_api_version='2', passcode='', password='***', profile='', > project_domain_id='', project_domain_name='default', project_id='', > project_name='admin', protocol='', redirect_uri='', region_name='', > remote_project_domain_id='', remote_project_domain_name='', > remote_project_id='', remote_project_name='', roles='', > service_provider='', service_provider_endpoint='', > service_provider_entity_id='', system_scope='', timing=False, token='***', > trust_id='', url='', user='', user_domain_id='', > user_domain_name='default', user_id='', username='adminapi', > verbose_level=3, verify=None) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > defaults: {u'auth_type': 'password', u'status': u'active', > u'image_status_code_retries': 5, > u'baremetal_introspection_status_code_retries': 5, 'api_timeout': None, > 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': > u'neutron', 'key': None, u'interface': u'public', u'network_api_version': > u'2', u'message': u'', u'image_format': u'qcow2', > u'baremetal_status_code_retries': 5, 'verify': True, 'cert': None, > u'secgroup_source': u'neutron', u'object_store_api_version': u'1', > u'disable_vendor_agent': {}} > cloud cfg: {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'orchestration_api_version': '1', > 'database_api_version': '1', 'metrics_api_version': '1', > 'data_processing_api_version': '1.1', 'auth_url': ' > http://controller:5000/v3', u'network_api_version': u'2', u'message': > u'', u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, > u'floating_ip_source': u'neutron', 'key': None, 'timing': False, > 'password': '***', 'cacert': None, 'key_manager_api_version': '1', > 'workflow_api_version': '2', u'baremetal_status_code_retries': '5', > 'identity_api_version': '3', 'deferred_help': False, 'cert': None, > u'secgroup_source': u'neutron', u'status': u'active', > 'alarming_api_version': '2', 'container_api_version': '1.20', u'interface': > 'public', u'disable_vendor_agent': {}} > compute API version 2.1, cmd group openstack.compute.v2 > network API version 2, cmd group openstack.network.v2 > image API version 2, cmd group openstack.image.v2 > volume API version 2, cmd group openstack.volume.v2 > identity API version 3, cmd group openstack.identity.v3 > object_store API version 1, cmd group openstack.object_store.v1 > container API version 1.20, cmd group openstack.container.v1 > messaging API version 2, cmd group openstack.messaging.v2 > database API version 1, cmd group openstack.database.v1 > data_processing API version 1.1, cmd group openstack.data_processing.v1 > load_balancer API version 2.0, cmd group openstack.load_balancer.v2 > neutronclient API version 2, cmd group openstack.neutronclient.v2 > workflow_engine API version 2, cmd group openstack.workflow_engine.v2 > container_infra API version 1, cmd group openstack.container_infra.v1 > orchestration API version 1, cmd group openstack.orchestration.v1 > dns API version 2, cmd group openstack.dns.v2 > key_manager API version 1, cmd group openstack.key_manager.v1 > metric API version 1, cmd group openstack.metric.v1 > alarming API version 2, cmd group openstack.alarming.v2 > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', u'network_api_version': u'2', > u'message': u'', u'image_format': u'qcow2', 'networks': [], > 'queues_api_version': '2', 'verify': True, 'dns_api_version': '2', > u'object_store_api_version': u'1', 'username': 'adminapi', > 'container_infra_api_version': '1', 'loadbalancer_api_version': '2.0', > 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > command: appcontainer exec -> zunclient.osc.v1.containers.ExecContainer > (auth=True) > Auth plugin password selected > auth_config_hook(): {'auth_type': 'password', 'beta_command': False, > u'image_status_code_retries': '5', 'key': None, 'database_api_version': > '1', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', > 'auth_url': 'http://controller:5000/v3', 'additional_user_agent': > [('osc-lib', '1.12.1')], u'network_api_version': u'2', u'message': u'', > u'image_format': u'qcow2', 'networks': [], 'queues_api_version': '2', > 'verify': True, 'dns_api_version': '2', u'object_store_api_version': u'1', > 'username': 'adminapi', 'container_infra_api_version': '1', > 'loadbalancer_api_version': '2.0', 'verbose_level': 3, 'region_name': '', > u'baremetal_introspection_status_code_retries': '5', 'api_timeout': None, > 'image_api_version': '2', 'auth': {'user_domain_name': 'default', > 'project_name': 'admin', 'project_domain_name': 'default'}, > 'default_domain': 'default', 'container_api_version': '1.20', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > 'orchestration_api_version': '1', 'timing': False, 'password': '***', > 'cacert': None, 'key_manager_api_version': '1', 'workflow_api_version': > '2', u'baremetal_status_code_retries': '5', 'identity_api_version': '3', > 'deferred_help': False, 'cert': None, u'secgroup_source': u'neutron', > u'status': u'active', 'alarming_api_version': '2', 'debug': True, > u'interface': 'public', u'disable_vendor_agent': {}} > Using auth plugin: password > Using parameters {'username': 'adminapi', 'project_name': 'admin', > 'user_domain_name': 'default', 'auth_url': 'http://controller:5000/v3', > 'password': '***', 'project_domain_name': 'default'} > Get auth_ref > REQ: curl -g -i -X GET http://controller:5000/v3 -H "Accept: > application/json" -H "User-Agent: openstacksdk/0.28.0 keystoneauth1/3.14.0 > python-requests/2.22.0 CPython/2.7.15rc1" > Starting new HTTP connection (1): controller:5000 > http://controller:5000 "GET /v3 HTTP/1.1" 200 250 > RESP: [200] Connection: Keep-Alive Content-Length: 250 Content-Type: > application/json Date: Sun, 02 Jun 2019 00:35:55 GMT Keep-Alive: timeout=5, > max=100 Server: Apache/2.4.29 (Ubuntu) Vary: X-Auth-Token > x-openstack-request-id: req-943ebe3a-aef8-4b96-8874-2727410e31c7 > RESP BODY: {"version": {"status": "stable", "updated": > "2018-10-15T00:00:00Z", "media-types": [{"base": "application/json", > "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.11", > "links": [{"href": "http://controller:5000/v3/", "rel": "self"}]}} > GET call to http://controller:5000/v3 used request id > req-943ebe3a-aef8-4b96-8874-2727410e31c7 > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-02T04:35:56.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["2yQsgqMjRMaIPXiDC0VwlA"], "issued_at": "2019-06-02T00:35:56.000000Z"}} > run(Namespace(command=[u'/bin/bash'], container=u'container2', > interactive=True)) > take_action(Namespace(command=[u'/bin/bash'], container=u'container2', > interactive=True)) > Instantiating zun client: <class 'zunclient.v1.client.Client'> > Making authentication request to http://controller:5000/v3/auth/tokens > http://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 11669 > {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": > "f09279600c8d4b019fd97b716f20e0b9", "name": "heat_stack_owner"}, {"id": > "caba611a1da84eab979aaa63a1aa240d", "name": "admin"}, {"id": > "d2fa56a7f6114a9da4ebbf9f3b52bb43", "name": "user"}], "expires_at": > "2019-06-02T04:35:56.000000Z", "project": {"domain": {"id": > "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, "id": > "75e35a49fbbe4c049b12ac80a6f21caa", "name": "admin"}, "catalog": > [{"endpoints": [{"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a128816d407542ee836bd30d8c36484d"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "cc381443f19d4e40829897e5e56a350c"}, {"url": " > http://controller:8386/v1.1/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d7395d22f01741d0ab1548585b06cd9c"}], "type": "data-processing", > "id": "01f448b630fd4eaf93645b7f659ac23e", "name": "sahara"}, {"endpoints": > [{"url": "http://controller:8774/v2.1", "interface": "internal", > "region": "RegionOne", "region_id": "RegionOne", "id": > "277e5b171ebf48f2b23c8974652b3346"}, {"url": "http://controller:8774/v2.1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "b7f50ef8b46d4b5a8f46926a7d58b1cd"}, {"url": " > http://controller:8774/v2.1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "c7a630944b0a4f1fb257cfa451012821"}], "type": "compute", "id": > "082e9aeda7c547f2bb1b7e59288b61c1", "name": "nova"}, {"endpoints": [{"url": > "http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "01e4ca0709444b768c498e9e7e8ef883"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3b4eef8ae0d64c1396386a5557d50560"}, {"url": " > http://controller:8776/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "af3e0786a82f4ddf86983b7948cb6b86"}], "type": "volumev2", "id": > "0aae6119825a4cf989db100bf6702a40", "name": "cinderv2"}, {"endpoints": > [{"url": "http://controller:9001/v2", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "881e3d793081405894b947eef1c926c6"}, {"url": "http://controller:9001/v2", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9e57d00fd56b4192a03c361d8dabf189"}, {"url": " > http://controller:9001/v2", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "b94c6a61349d4afa9bdef07983ff5caf"}], "type": "dns", "id": > "1c223d100bb34b268cd7ce1317a12df1", "name": "designate"}, {"endpoints": > [{"url": "http://controller:9292", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "1442dd1770bb45fea5cf478291113ea0"}, {"url": "http://controller:9292", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "6bed83d9ffce4ba5943b8fce7193c2e7"}, {"url": "http://controller:9292", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "fb9ad4c2c6cb4ed5b99217ef0a8037b3"}], "type": "image", "id": > "25970e290a4b4faabfffa38acf3466a6", "name": "glance"}, {"endpoints": > [{"url": "http://controller:5000/v3", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "311fe40608f94104b5d5281acc58ed62"}, {"url": "http://controller:5000/v3", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "92dbbc67188a4689a2566765cb881e71"}, {"url": " > http://controller:5000/v3", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "e8d3798af06f4ab7951052c0728291e5"}], > "type": "identity", "id": "27040e80b018468daa25005e358da1fb", "name": > "keystone"}, {"endpoints": [{"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "3827d192568f4b1e8b1901bdedbe1137"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "916be6147a2d4df293ebf375abd6fed2"}, {"url": " > http://controller:8786/v2/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c7fbcbf8a16d4e48a6f31ba79c194f56"}], "type": "sharev2", "id": > "444f1ba4171b47e088c2b3fd8e0637e3", "name": "manilav2"}, {"endpoints": > [{"url": "http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "1ead177408974a40a500ffc5f92acb8c"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "56aebd20ee19475cae24f2e558de4b2d"}, {"url": " > http://controller:8776/v3/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "f418bdb5891042148aa73518e431d31d"}], "type": "volumev3", "id": > "589bffafa24d417fbf6b91e3cb03822a", "name": "cinderv3"}, {"endpoints": > [{"url": "http://controller:9511/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "2d9916f43f4c45418cc0499ac81bf0b5"}, {"url": "http://controller:9511/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "33141d1c546a4bbd96971d7116d7bcfa"}, {"url": " > http://controller:9511/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "e5a85e4582d04fa19e597774c2b4b725"}], > "type": "container-infra", "id": "6015ada3d74d4196808ef0f0547506af", > "name": "magnum"}, {"endpoints": [{"url": " > http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "52504619399d467ba64a1f232129768c"}, {"url": " > http://controller:8080/v1", "interface": "admin", "region": "RegionOne", > "region_id": "RegionOne", "id": "8db772741fc04adba7dbb516d256232a"}, > {"url": "http://controller:8080/v1/AUTH_75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "a0a947a9b7e44a29a268cbaa6f438302"}], "type": "object-store", "id": > "7ae555fa260c415eacae5050b58453f2", "name": "swift"}, {"endpoints": > [{"url": "http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "0dde27f774ce4c43b7de3112f2964386"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "9f4f7f5839a3449da8554c732582f676"}, {"url": " > http://controller:8779/v1.0/75e35a49fbbe4c049b12ac80a6f21caa", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "c3f012574dd9428d9e35f03c6dec0576"}], "type": "database", "id": > "7df0f5dd1aec42dd8d9d5c89c77ead16", "name": "trove"}, {"endpoints": > [{"url": "http://controller:8041", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "0018cbfa02c848a69eb8ebf34a15247c"}, {"url": "http://controller:8041", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "03edf63220d04121929f7aa6061076b8"}, {"url": "http://controller:8041", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "703d57680cb3441aa91cd5554f254237"}], "type": "metric", "id": > "80706a8ee7414e8e94384a1052a525f1", "name": "gnocchi"}, {"endpoints": [], > "type": "volume", "id": "88b625d307d04657bb7b011d502cb105", "name": > "cinder"}, {"endpoints": [{"url": "http://controller:9696", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "30bd1431591349faaec6cde3d7b6235a"}, {"url": "http://controller:9696", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "75569f0dd24e44f2bb43c071b31a5b4d"}, {"url": "http://controller:9696", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "d92693470d9f41a2ac8e72809cb281e2"}], "type": "network", "id": > "97130e1d51984b36b06a0038d7217da1", "name": "neutron"}, {"endpoints": > [{"url": "http://controller:8778", "interface": "public", "region": > "RegionOne", "region_id": "RegionOne", "id": > "a419376d73e048daaa4235384ad0a4fc"}, {"url": "http://controller:8778", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "d04944458ffc46eb8e1bb3ccdd185b69"}, {"url": "http://controller:8778", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "d8f093d52e5a47c499ac7d1d4f8a82c3"}], "type": "placement", "id": > "a69c9d9153e3484a849cfd78dc1fab56", "name": "placement"}, {"endpoints": [], > "type": "metering", "id": "aa617d65d2624a7caf065ebdef4485c5", "name": > "ceilometer"}, {"endpoints": [{"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "0145b180de764baabb3f8fc15e5e0883"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "b6006479263b4faf81963f21f523e11d"}, {"url": " > http://controller:8786/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "c1214706df5648c5add384083c0e787e"}], "type": "share", "id": > "ac0fa1b7b9284b909918e231bd7f4d77", "name": "manila"}, {"endpoints": > [{"url": "http://controller:8000/v1", "interface": "admin", "region": > "RegionOne", "region_id": "RegionOne", "id": > "6ca40f05d84f4fa8886befc3e24b538a"}, {"url": "http://controller:8000/v1", > "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", > "id": "7b4e560125674762b7d516605986f5aa"}, {"url": " > http://controller:8000/v1", "interface": "public", "region": "RegionOne", > "region_id": "RegionOne", "id": "dd8019b5e518452bad5dab7bd1b8ea50"}], > "type": "cloudformation", "id": "dcd80af5964c4766986e92e38e0b6920", "name": > "heat-cfn"}, {"endpoints": [{"url": "http://controller:9517/v1", > "interface": "public", "region": "RegionOne", "region_id": "RegionOne", > "id": "04d221684116407981f9d00d177b077a"}, {"url": " > http://controller:9517/v1", "interface": "internal", "region": > "RegionOne", "region_id": "RegionOne", "id": > "4dc7c4de0dfd44cdaa3e74598795d8c9"}, {"url": "http://controller:9517/v1", > "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", > "id": "a71ac55b3dd94bee9bd4b7d058b0d49d"}], "type": "container", "id": > "e033c90bdd484ff79e246ac81750e767", "name": "zun"}, {"endpoints": [{"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "internal", "region": "RegionOne", "region_id": "RegionOne", "id": > "1253fe5f4dcd42fd8f20e56c8fc14c1c"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "public", "region": "RegionOne", "region_id": "RegionOne", "id": > "848b8262069a4c4c91716c52644783d8"}, {"url": " > http://controller:8004/v1/75e35a49fbbe4c049b12ac80a6f21caa", "interface": > "admin", "region": "RegionOne", "region_id": "RegionOne", "id": > "e7c4c75e7d614be3ab85fe0fade29eeb"}], "type": "orchestration", "id": > "fd3d4a5666aa42ddadc5783a156c230d", "name": "heat"}], "user": {"domain": > {"id": "93406f7e7b4a44a3af7b9bfca3f42c66", "name": "default"}, > "password_expires_at": null, "name": "adminapi", "id": > "f59c50813bb24aad9c7a61a3392a9805"}, "audit_ids": > ["vrRr0wvdRnOG0uM2hl19_A"], "issued_at": "2019-06-02T00:35:56.000000Z"}} > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container2/execute?command=%22%2Fbin%2Fbash%22&run=False&interactive=True > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6cd9644ef619ef1d55102a08eabcacc7e649405e31ceb7102219cf77e6951a50" > Starting new HTTP connection (1): controller:9517 > http://controller:9517 "POST > /v1/containers/container2/execute?command=%22%2Fbin%2Fbash%22&run=False&interactive=True > HTTP/1.1" 200 310 > RESP: [200] Connection: keep-alive Content-Length: 310 Content-Type: > application/json Date: Sun, 02 Jun 2019 00:35:57 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-fc701b80-a45f-4216-8790-c32c58190273 > RESP BODY: {"output": null, "exec_id": > "b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa", > "proxy_url": > "ws://controller:6784/?token=bd15f3a6-feb1-47f9-befe-6793a6c017eb&uuid=93acec9f-0234-44d9-a796-6e06e31fc90c&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa", > "exit_code": null} > POST call to container for > http://controller:9517/v1/containers/container2/execute?command=%22%2Fbin%2Fbash%22&run=False&interactive=True > used request id req-fc701b80-a45f-4216-8790-c32c58190273 > connecting to: > ws://controller:6784/?token=bd15f3a6-feb1-47f9-befe-6793a6c017eb&uuid=93acec9f-0234-44d9-a796-6e06e31fc90c&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa > connected to container2, press Enter to continue > type ~. to disconnect > REQ: curl -g -i -X POST > http://controller:9517/v1/containers/container2/execute_resize?h=36&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa&w=142 > -H "Accept: application/json" -H "Content-Length: 0" -H "Content-Type: > application/json" -H "OpenStack-API-Version: container 1.20" -H > "User-Agent: None" -H "X-Auth-Token: > {SHA256}6cd9644ef619ef1d55102a08eabcacc7e649405e31ceb7102219cf77e6951a50" > http://controller:9517 "POST > /v1/containers/container2/execute_resize?h=36&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa&w=142 > HTTP/1.1" 500 261 > RESP: [500] Connection: keep-alive Content-Length: 261 Content-Type: > application/json Date: Sun, 02 Jun 2019 00:36:07 GMT > Openstack-Api-Maximum-Version: container 1.22 > Openstack-Api-Minimum-Version: container 1.1 Openstack-Api-Version: > container 1.20 Vary: OpenStack-API-Version X-Openstack-Request-Id: > req-d8514136-a496-417b-b202-e4b63ee49d9b > RESP BODY: {"errors": [{"status": 500, "code": "container.", "links": [], > "title": "Internal Server Error", "detail": "Your request could not be > handled because of a problem in the server. Error Correlation id is: > e873667c-cfc6-495a-97c2-86411845836d", "request_id": ""}]} > POST call to container for > http://controller:9517/v1/containers/container2/execute_resize?h=36&exec_id=b749c0d0d4b31beea7f2c7577a16ccbbd45bdb07ea7834b651a575a3be12f3aa&w=142 > used request id req-d8514136-a496-417b-b202-e4b63ee49d9b > Internal Server Error (HTTP 500) (Request-ID: > req-d8514136-a496-417b-b202-e4b63ee49d9b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-d8514136-a496-417b-b202-e4b63ee49d9b) > clean_up ExecContainer: Internal Server Error (HTTP 500) (Request-ID: > req-d8514136-a496-417b-b202-e4b63ee49d9b) > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 136, in run > ret_val = super(OpenStackShell, self).run(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 281, in > result = self.run_subcommand(remainder) > File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line > 176, in run_subcommand > ret_value = super(OpenStackShell, self).run_subcommand(argv) > File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 401, in > run_subcommand > result = cmd.run(parsed_args) > "/usr/local/lib/python2.7/dist-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line > 184, in run > return_code = self.take_action(parsed_args) or 0 > "/usr/local/lib/python2.7/dist-packages/zunclient/osc/v1/containers.py", > line 537, in take_action > websocketclient.do_exec(client, url, container, exec_id, "~", 0.5) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 379, in do_exec > wscls.handle_resize() > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 196, in handle_resize > self.tty_resize(height=rows, width=cols) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/websocketclient/websocketclient.py", > line 294, in tty_resize > self.cs.containers.execute_resize(self.id, self.exec_id, width, > height) > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 165, in execute_resize > qparams={'exec_id': exec_id, 'w': width, 'h': height})[1] > "/usr/local/lib/python2.7/dist-packages/zunclient/v1/containers.py", line > 128, in _action > **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 367, in json_request > resp = self._http_request(url, method, **kwargs) > "/usr/local/lib/python2.7/dist-packages/zunclient/common/httpclient.py", > line 350, in _http_request > error_json.get('debuginfo'), method, url) > InternalServerError: Internal Server Error (HTTP 500) (Request-ID: > req-d8514136-a496-417b-b202-e4b63ee49d9b) > END return value: 1 > You received this bug notification because you are subscribed to Zun. > Matching subscriptions: zun > https://bugs.launchpad.net/bugs/1830450 > Title: > openstack appcontainer exec --interactive > Status in Zun: > Bug description: > I have this error when I exec intercatve container > # openstack appcontainer exec --interactive container /bin/sh > HTTPConnectionPool(host='130.127.133.20', port=2375): Max retries > exceeded with url: > /v1.35/exec/808ebed1841484c028d0f0684f7637fe28849592125607c55173155ca0f47668/start > (Caused by NewConnectionError('<urllib3.connection.HTTPConnection > object at 0x7f44ae9ef550>: Failed to establish a new connection: > [Errno 111] Connection refused',)) > /etc/systemd/system/docker.service.d/docker.conf #my compute node > [Service] > ExecStart= > ExecStart=/usr/bin/dockerd --group zun -H tcp://compute-1:2375 -H > unix:///var/run/docker.sock --cluster-store etcd://controller:2379 > To manage notifications about this bug go to: > https://bugs.launchpad.net/zun/+bug/1830450/+subscriptions <div class="comment-text editable-message-text" itemprop="commentText"><p>Sorry, I missed something in before.</p> <p>I went through your zun-wsproxy log again and saw this message:</p> <p>May 27 10:09:07 controller.<wbr/>zunrocky.<wbr/>labs1-pg0.<wbr/>clemson.<wbr/>cloudlab.<wbr/>us zun-wsproxy[26330]: 2019-05-27 10:09:07.417 47902 INFO zun.websocket.<wbr/>websocketproxy [req-15d4d06e-<wbr/>2be6-49e5-<wbr/>bc2e-72f14b4341<wbr/>a9 - - - - -] handler exception: HTTPConnectionP<wbr/>ool(host=<wbr/>'130.127.<wbr/>133.152'<wbr/>, port=2375): Max retries exceeded with url: /v1.30/<wbr/>exec/03f04710c6<wbr/>e03154e66aba390<wbr/>979e269a72defe6<wbr/>19c67e7318839f6<wbr/>705e8de4e/<wbr/>start (Caused by NewConnectionEr<wbr/>ror('<urllib3.<wbr/>connection.<wbr/>HTTPConnection object at 0x7f02a3752890>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',))</p> <p>It means your zun-wsproxy couldn't connect to your Docker Daemon to start the exec session. You might want to check if there is a firewall (or anything else) that block the traffic from zun-wsproxy (in controller node) to the Docker Daemon (listen to <compute_<wbr/>node_ip_<wbr/>address><wbr/>:2375).</p></div> <textarea style="width: 100%" rows="10">Sorry, I missed something in before. I went through your zun-wsproxy log again and saw this message: May 27 10:09:07 controller.zunrocky.labs1-pg0.clemson.cloudlab.us zun-wsproxy[26330]: 2019-05-27 10:09:07.417 47902 INFO zun.websocket.websocketproxy [req-15d4d06e-2be6-49e5-bc2e-72f14b4341a9 - - - - -] handler exception: HTTPConnectionPool(host='130.127.133.152', port=2375): Max retries exceeded with url: /v1.30/exec/03f04710c6e03154e66aba390979e269a72defe619c67e7318839f6705e8de4e/start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02a3752890>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',)) It means your zun-wsproxy couldn't connect to your Docker Daemon to start the exec session. You might want to check if there is a firewall (or anything else) that block the traffic from zun-wsproxy (in controller node) to the Docker Daemon (listen to <compute_node_ip_address>:2375). <div class="comment-text editable-message-text" itemprop="commentText"><p>FWIW, here is how interactive exec work in Rocky or later version:</p> <p>1. Client send request to Zun to create an exec session in docker (client -> zun-api -> zun-compute -> docker)<br/> 2. Client send request to start an exec session (client -> zun-wsproxy -> docker)<br/> 3. Client resize the exec session (client -> zun-api -> zun-compute -> docker)</p> <p>In your case, it looks #2 (zun-wsproxy cannot connect to docker) is broken.</p></div> <textarea style="width: 100%" rows="10">FWIW, here is how interactive exec work in Rocky or later version: 1. Client send request to Zun to create an exec session in docker (client -> zun-api -> zun-compute -> docker) 2. Client send request to start an exec session (client -> zun-wsproxy -> docker) 3. Client resize the exec session (client -> zun-api -> zun-compute -> docker) In your case, it looks #2 (zun-wsproxy cannot connect to docker) is broken. As reported by https://bugs.launchpad.net/zun/+bug/1831506 (comment #8), this problem is resolved by configuring 'docker_remote_api_url'. I am going to close this bug. I use stable/victora zun, when i run zun exec --interactive container2 /bin/sh it gave an error: Failed to connect to remote host:[Errno 111] Connection refused:container2 when i run zun --debug run -i cirros /bin/sh, the debug output is http://paste.openstack.org/show/802032/ i can run docker exec command but i cant run zun exec command, do you have any idea? This error looks odd: DEBUG (websocketclient:240) connecting to: ws://0.0.0.0:6784/?token=d9e36478-1a52-4a9d-8046-c4d6ed80e5c2&uuid=2be6eb3d-708e-4997-a827-8d2294bfd6cb It should not connect to 0.0.0.0. Could you check the following options in your config file? [websocket_proxy] base_url wsproxy_host