Le 2013-11-29 16:42, Aurelien Bompard a écrit :
Yeah the configuration key in your mailman config file is wrong. It should not point to the django config file directly, it should point to another config file with this content:
[general] base_url: http://yourserver.example.com/hyperkitty django_settings: /home/mailman/hyperkitty_standalone/settings.py
Did you follow some documentation which said to put the path to the django settings file there? Because that's how it used to work some months ago, and I may have forgotten to update something.
Aurélien
ok, changed to some correct values, but still unusable.
exploring quickly mailman, i was at rev 7229 which brings recent changes on archiving (7228 allows per-list archiver, quite a new feature and maybe a bit unstable) went back to beta3 for testing, but still not archiving
Which mailman version do you use ?
On Nov 30, 2013, at 06:42 PM, nicolas wrote:
exploring quickly mailman, i was at rev 7229 which brings recent changes on archiving (7228 allows per-list archiver, quite a new feature and maybe a bit unstable)
Hopefully not. It's fairly well covered with unit tests. The biggest question is whether the API is good and useful. Joanna's original branch put per-list archiving configuration on the <root>/3.0/lists/<mylist>/config resource, but this didn't make sense to me because you probably want to PUT and PATCH the resource to enable and disable individual archivers for a list. So I created a new resource <root>/3.0/lists/<mylist>/archivers which is used to manage the per-list archivers.
The other change I made was to by default enable any per-list archiver which is enabled site-wide, and to make all the defined site-wide archivers available to every mailing list, regardless of whether the archiver is enabled site-wide or not. For archivers which are disabled site-wide, you can still set the per-list archiver state, but the site setting takes precedence.
Cheers, -Barry
I think we have to qualify a better report before sending to mailman team, i'm using r7229 of lp:mailman, and i wrote this simple mlist.py module : #-*- coding: utf-8 -*-
from mailman.interfaces.mailinglist import IListArchiverSet
def showArchiver(mlist): "" archiver_set = IListArchiverSet(mlist) # new feature of http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/revision/7228
then i'm trying to run it :
(py27)mailman@mail:~$ mailman withlist -r mlist.showArchiver mylist Traceback (most recent call last): File "/home/mailman/py27/bin/mailman", line 9, in <module> load_entry_point('mailman==3.0.0b3-', 'console_scripts', 'mailman')() File "/home/mailman/py27/local/lib/python2.7/site-packages/mailman-3.0.0b3_-py2.7.egg/mailman/bin/mailman.py", line 98, in main args.func(args) File "/home/mailman/py27/local/lib/python2.7/site-packages/mailman-3.0.0b3_-py2.7.egg/mailman/commands/cli_withlist.py", line 134, in process r = call_name(dotted_name, m) File "/home/mailman/py27/local/lib/python2.7/site-packages/mailman-3.0.0b3_-py2.7.egg/mailman/utilities/modules.py", line 65, in call_name return named_callable(*args, **kws) File "/home/mailman/mlist.py", line 7, in showArchiver archiver_set = IListArchiverSet(mlist) File "/home/mailman/py27/local/lib/python2.7/site-packages/zope.component-4.1.0-py2.7.egg/zope/component/hookable.py", line 33, in __call__ return self.__implementation(*args, **kw) File "/home/mailman/py27/local/lib/python2.7/site-packages/zope.component-4.1.0-py2.7.egg/zope/component/_api.py", line 156, in adapter_hook return sitemanager.queryAdapter(object, interface, name, default) File "/home/mailman/py27/local/lib/python2.7/site-packages/zope.interface-4.0.5-py2.7-linux-x86_64.egg/zope/interface/registry.py", line 229, in queryAdapter return self.adapters.queryAdapter(object, interface, name, default) File "/home/mailman/py27/local/lib/python2.7/site-packages/mailman-3.0.0b3_-py2.7.egg/mailman/model/mailinglist.py", line 590, in __init__ ListArchiver.name == archiver_name)).one() File "/home/mailman/py27/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-x86_64.egg/storm/expr.py", line 412, in __eq__ other = getattr(self, "variable_factory", Variable)(value=other) File "/home/mailman/py27/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-x86_64.egg/storm/variables.py", line 396, in parse_set % (type(value), value)) TypeError: Expected unicode, found <type 'str'>: 'hyperkitty'
This tends to be confirmed by conf dump : (py27)mailman@mail:~$ mailman conf [logging.archiver] path: archiver.log [logging.archiver] datefmt: %b %d %H:%M:%S %Y [logging.archiver] format: %(asctime)s (%(process)d) %(message)s [logging.archiver] propagate: no [logging.archiver] level: debug [logging.locks] path: mailman.log [logging.locks] datefmt: %b %d %H:%M:%S %Y [logging.locks] format: %(asctime)s (%(process)d) %(message)s [logging.locks] propagate: no [logging.locks] level: info [passwords] configuration: python:mailman.config.passlib [passwords] password_length: 8 [logging.mischief] path: mailman.log [logging.mischief] datefmt: %b %d %H:%M:%S %Y [logging.mischief] format: %(asctime)s (%(process)d) %(message)s [logging.mischief] propagate: no [logging.mischief] level: info [mailman] filtered_messages_are_preservable: no [mailman] post_hook: [mailman] layout: dev [mailman] default_language: en [mailman] pending_request_life: 3d [mailman] noreply_address: noreply [mailman] site_owner: owner [mailman] sender_headers: from from_ reply-to sender [mailman] pre_hook: [mailman] email_commands_max_lines: 10 [archiver.prototype] enable: no [archiver.prototype] clobber_date: maybe [archiver.prototype] class: mailman.archiving.prototype.Prototype [archiver.prototype] configuration: changeme [archiver.prototype] clobber_skew: 1d [devmode] recipient: [devmode] testing: no [devmode] enabled: no [devmode] wait: 10s [archiver.mail_archive] enable: no [archiver.mail_archive] clobber_date: maybe [archiver.mail_archive] class: mailman.archiving.mailarchive.MailArchive [archiver.mail_archive] configuration: python:mailman.config.mail_archive [archiver.mail_archive] clobber_skew: 1d [logging.config] path: config.log [logging.config] datefmt: %b %d %H:%M:%S %Y [logging.config] format: %(asctime)s (%(process)d) %(message)s [logging.config] propagate: no [logging.config] level: debug [antispam] header_checks: [antispam] jump_chain: hold [digests] mime_digest_keep_headers: Date From To Cc Subject Message-ID Keywords In-Reply-To References Content-Type MIME-Version Content-Transfer-Encoding Precedence Reply-To Message List-Post [digests] plain_digest_keep_headers: Message Date From Subject To Cc Message-ID Keywords Content-Type [styles] default: legacy-default [styles] paths: mailman.styles [shell] use_ipython: no [shell] banner: Welcome to the GNU Mailman shell [shell] prompt: >>> [logging.error] path: mailman.log [logging.error] datefmt: %b %d %H:%M:%S %Y [logging.error] format: %(asctime)s (%(process)d) %(message)s [logging.error] propagate: no [logging.error] level: info [logging.smtp] datefmt: %b %d %H:%M:%S %Y [logging.smtp] success: $msgid post to $listname from $sender, $size bytes [logging.smtp] format: %(asctime)s (%(process)d) %(message)s [logging.smtp] level: info [logging.smtp] refused: $msgid post to $listname from $sender, $size bytes, $refused failures [logging.smtp] failure: $msgid delivery to $recip failed with code $smtpcode, $smtpmsg [logging.smtp] every: $msgid smtp to $listname for $recip recips, completed in $time seconds [logging.smtp] path: smtp.log [logging.smtp] propagate: no [logging.http] path: mailman.log [logging.http] datefmt: %b %d %H:%M:%S %Y [logging.http] format: %(asctime)s (%(process)d) %(message)s [logging.http] propagate: no [logging.http] level: info [logging.root] path: mailman.log [logging.root] datefmt: %b %d %H:%M:%S %Y [logging.root] format: %(asctime)s (%(process)d) %(message)s [logging.root] propagate: no [logging.root] level: info [logging.fromusenet] path: mailman.log [logging.fromusenet] datefmt: %b %d %H:%M:%S %Y [logging.fromusenet] format: %(asctime)s (%(process)d) %(message)s [logging.fromusenet] propagate: no [logging.fromusenet] level: info [logging.bounce] path: bounce.log [logging.bounce] datefmt: %b %d %H:%M:%S %Y [logging.bounce] format: %(asctime)s (%(process)d) %(message)s [logging.bounce] propagate: no [logging.bounce] level: info [logging.vette] path: mailman.log [logging.vette] datefmt: %b %d %H:%M:%S %Y [logging.vette] format: %(asctime)s (%(process)d) %(message)s [logging.vette] propagate: no [logging.vette] level: info [webservice] admin_pass: restpass [webservice] admin_user: restadmin [webservice] show_tracebacks: yes [webservice] hostname: localhost [webservice] api_version: 3.0 [webservice] port: 8001 [webservice] use_https: no [database] url: sqlite:///$DATA_DIR/mailman.db [database] debug: on [database] migrations_path: mailman.database.schema [database] class: mailman.database.sqlite.SQLiteDatabase [nntp] rewrite_duplicate_headers: To X-Original-To CC X-Original-CC Content-Transfer-Encoding X-Original-Content-Transfer-Encoding MIME-Version X-MIME-Version [nntp] port: [nntp] host: [nntp] user: [nntp] password: [nntp] remove_headers: nntp-posting-host nntp-posting-date x-trace x-complaints-to xref date-received posted posting-version relay-version received [logging.runner] path: runner.log [logging.runner] datefmt: %b %d %H:%M:%S %Y [logging.runner] format: %(asctime)s (%(process)d) %(message)s [logging.runner] propagate: no [logging.runner] level: debug [bounces] register_bounces_every: 15m [logging.subscribe] path: mailman.log [logging.subscribe] datefmt: %b %d %H:%M:%S %Y [logging.subscribe] format: %(asctime)s (%(process)d) %(message)s [logging.subscribe] propagate: no [logging.subscribe] level: info [logging.debug] path: debug.log [logging.debug] datefmt: %b %d %H:%M:%S %Y [logging.debug] format: %(asctime)s (%(process)d) %(message)s [logging.debug] propagate: no [logging.debug] level: info [mta] verp_confirm_regexp: ^(.*<)?(?P<addr>[^+]+?)+(?P<cookie>[^@]+)@.*$ [mta] verp_confirm_format: $address+$cookie [mta] lmtp_port: 8024 [mta] verp_probe_format: $bounces+$token@$domain [mta] verp_probe_regexp: ^(?P<bounces>[^+]+?)+(?P<token>[^@]+)@.*$ [mta] verp_regexp: ^(?P<bounces>[^+]+?)+(?P<local>[^=]+)=(?P<domain>[^@]+)@.*$ [mta] max_recipients: 500 [mta] verp_personalized_deliveries: no [mta] max_sessions_per_connection: 0 [mta] smtp_port: 25 [mta] smtp_pass: [mta] max_autoresponses_per_day: 10 [mta] outgoing: mailman.mta.deliver.deliver [mta] delivery_retry_period: 5d [mta] remove_dkim_headers: no [mta] smtp_host: mail.menestrel.local [mta] max_delivery_threads: 0 [mta] lmtp_host: mail.menestrel.local [mta] verp_format: ${bounces}+${local}=${domain} [mta] configuration: python:mailman.config.postfix [mta] verp_delivery_interval: 0 [mta] incoming: mailman.mta.postfix.LMTP [mta] verp_delimiter: + [mta] smtp_user: [mta] verp_probes: no [mta] verp_confirmations: no [archiver.mhonarc] enable: no [archiver.mhonarc] clobber_date: maybe [archiver.mhonarc] class: mailman.archiving.mhonarc.MHonArc [archiver.mhonarc] configuration: python:mailman.config.mhonarc [archiver.mhonarc] clobber_skew: 1d
while i have var/etc/mailman.cfg is :
(py27)mailman@mail:~$ more var/etc/mailman.cfg # AUTOMATICALLY GENERATED BY MAILMAN ON 2013-11-26 22:27:51 # # This is your GNU Mailman 3 configuration file. You can edit this file to # configure Mailman to your needs, and Mailman will never overwrite it. # Additional configuration information is (for now) available in the # schema.cfg file http://tinyurl.com/cm5rtqe and the base mailman.cfg file # http://tinyurl.com/dx9b8eg. # # For example, uncomment the following lines to run Mailman in developer mode. # #[devmode] #enabled: yes #recipient: your.address@your.domain [mailman] site_owner: owner #default_language: fr [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: mail.menestrel.local lmtp_port: 8024 smtp_host: mail.menestrel.local smtp_port: 25
[postfix] # Additional configuration variables for the postfix MTA.
# This variable describe the program to use for regenerating the transport map # db file, from the associated plain text files. The file being updated will # be appended to this string (with a separating space), so it must be # appropriate for os.system(). postmap_command: /usr/sbin/postmap
[logging.archiver] format: %(asctime)s (%(process)d) %(message)s datefmt: %b %d %H:%M:%S %Y propagate: no level: debug path: archiver.log
[logging.config] level: debug path: config.log [logging.runner] level: debug path: runner.log
[archiver.hyperkitty] class: hyperkitty.archiver.Archiver enable: yes configuration: /home/mailman/hyperkitty_standalone/hyperkitty.cfg
[runner.lmtp] class: mailman.runners.lmtp.LMTPRunner path: /home/mailman/mailman/var/data/postfix_lmtp.db
See, archiver.hyperkitty didn't show up at mailman conf while some other custom conf is shown correctly...
can you reproduce this on your side ?
On Nov 30, 2013, at 08:37 PM, nicolas wrote:
I think we have to qualify a better report before sending to mailman team, i'm using r7229 of lp:mailman, and i wrote this simple mlist.py module : #-*- coding: utf-8 -*-
^^^ This isn't enough to make it work. MM3 core is written in a dialect that tries to be as close to Python 3 as possible[*]. You should put the following at the top of your module:
from __future__ import absolute_import, print_function, unicode_literals
It's possible there are some corner cases when using this as a withlist script. If so, please submit a bug report!
Cheers, -Barry
Oh, I forgot: I'm using patches on Mailman from all the bugs I've reported and are not yet merged:
- https://bugs.launchpad.net/mailman/+bug/1130957 - https://bugs.launchpad.net/mailman/+bug/1158721 - https://bugs.launchpad.net/mailman/+bug/1130696 - https://bugs.launchpad.net/mailman/+bug/1170347 - https://bugs.launchpad.net/mailman/+bug/1236297
All those have patches attached. There's also my work on the import21 command, but I don't think it would impact you.
This may be linked to why the archiver does not work for you. Cant you try with the paches?
Thanks. I should have thought of that earlier, sorry... :-/
Aurélien
P.S.: Hi Barry, cool to see you around :-)
2013/11/30 Barry Warsaw barry@python.org:
On Nov 30, 2013, at 08:37 PM, nicolas wrote:
I think we have to qualify a better report before sending to mailman team, i'm using r7229 of lp:mailman, and i wrote this simple mlist.py module : #-*- coding: utf-8 -*-
^^^ This isn't enough to make it work. MM3 core is written in a dialect that tries to be as close to Python 3 as possible[*]. You should put the following at the top of your module:
from __future__ import absolute_import, print_function, unicode_literals
It's possible there are some corner cases when using this as a withlist script. If so, please submit a bug report!
Cheers, -Barry
hyperkitty-devel mailing list hyperkitty-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/hyperkitty-devel
Hi,
Thanks to both of you. this was quite helpful
Aurelien, i've created a branch with patches (except the postgres related one)
https://code.launchpad.net/~nkarageuzian/mailman/hyperkitty_compliance
and now I have the following stack when i'm trying to release messages : Dec 01 14:46:57 2013 (22078) HOLD: newsletter@lists.menestrel.local post from nico@vostrodeb held, message-id=20131201134656.6BA28762BB@vostrodeb: n/a Dec 01 14:46:58 2013 (22078) OWNER: 20131201134657.22078.70812@vostrodeb Dec 01 14:47:01 2013 (22085) lists.menestrel.local - - "GET /3.0/lists/newsletter@lists.menestrel.local HTTP/1.1" 200 369 Traceback (most recent call last): File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run self.result = application(self.environ, self.start_response) File "/home/nico/wspace/mailman/src/mailman/database/transaction.py", line 60, in wrapper rtn = function(*args, **kws) File "/home/nico/wspace/mailman/src/mailman/rest/wsgiapp.py", line 59, in __call__ environ, start_response) File "/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/app.py", line 18, in __call__ response = self.get_response(request, resource_or_response) File "/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/app.py", line 65, in get_response resource_or_response = resource_or_response(request) File "/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/resource.py", line 212, in __call__ return _dispatch(request, match, lambda r: callable(self, r)) File "/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/resource.py", line 243, in _dispatch response = func(request) File "/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/resource.py", line 212, in <lambda> return _dispatch(request, match, lambda r: callable(self, r)) File "/home/nico/wspace/mailman/src/mailman/rest/moderation.py", line 162, in requests resource = self._make_collection(request) File "/home/nico/wspace/mailman/src/mailman/rest/helpers.py", line 180, in _make_collection for resource in collection] File "/home/nico/wspace/mailman/src/mailman/rest/moderation.py", line 151, in _resource_as_dict return self._make_resource(request.id) File "/home/nico/wspace/mailman/src/mailman/rest/moderation.py", line 86, in _make_resource msg = getUtility(IMessageStore).get_message_by_id(key) File "/home/nico/wspace/mailman/src/mailman/database/transaction.py", line 78, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/home/nico/wspace/mailman/src/mailman/model/messagestore.py", line 113, in get_message_by_id return self._get_message(row) File "/home/nico/wspace/mailman/src/mailman/model/messagestore.py", line 105, in _get_message with open(path) as fp: IOError: [Errno 2] No such file or directory: '/home/nico/wspace/mailman/var/messages/WR/TF/WRTFIZG5HYLEDRWDDT4PZDVBQZW26SOV' Dec 01 14:47:01 2013 (22085) lists.menestrel.local - - "GET /3.0/lists/newsletter@lists.menestrel.local/held HTTP/1.1" 500 59 and when i look in directory, there are no files.
That's some concern, but i have now some more hyperkitty specific (related to report about postgres i have in common with mayrin) Given the following script :
#-*- coding: utf-8 -*- from __future__ import absolute_import, print_function, unicode_literals
from mailman.interfaces.mailinglist import IListArchiverSet from mailman.testing.helpers import ( specialized_message_from_string as mfs) def showArchiver(mlist): "" archiver_set = IListArchiverSet(mlist) # new feature of http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/revision/7228 archiver = archiver_set.get(u'hyperkitty') print("%s - %s" % (mlist.list_name,archiver.name) ) msg = mfs("""\ To: %s From: anne@example.com Subject: Testing the test list Message-ID: <ant> X-Message-ID-Hash: MS6QLWERIJLGCRF44J7USBFDELMNT2BW
Tests are better than no tests but the water deserves to be swum. """ % mlist.list_name) print(archiver) archiver.system_archiver.archive_message(msg,mlist)
got the following output :
(venv_hk)nico@vostrodeb:~/wspace/mailman$ mailman withlist -r mlist.showArchiver newsletter@lists.menestrel.local newsletter - hyperkitty <mailman.model.mailinglist.ListArchiver object at 0x38fdad0> [09:12:37.936526] EXECUTE: 'SELECT * FROM patch WHERE version=0', () [09:12:37.938982] DONE [09:12:37.939733] EXECUTE: 'SELECT patch.version FROM patch', () [09:12:37.940496] DONE [09:12:37.943501] EXECUTE: 'SELECT patch.version FROM patch', () [09:12:37.944198] DONE
some long time ~ 3mn while we can observe : 23624 nico 20 0 307m 202m 5952 R 99,9 5,4 2:31.86 mailman 1 full CPU for mailman then shows up the stack : Traceback (most recent call last): File "/home/nico/wspace/venv_hk/bin/mailman", line 9, in <module> load_entry_point('mailman==3.0.0b3-', 'console_scripts', 'mailman')() File "/home/nico/wspace/mailman/src/mailman/bin/mailman.py", line 98, in main args.func(args) File "/home/nico/wspace/mailman/src/mailman/commands/cli_withlist.py", line 134, in process r = call_name(dotted_name, m) File "/home/nico/wspace/mailman/src/mailman/utilities/modules.py", line 65, in call_name return named_callable(*args, **kws) File "/home/nico/wspace/mlist.py", line 22, in showArchiver archiver.system_archiver.archive_message(msg,mlist) File "/home/nico/wspace/hyperkitty/hyperkitty/archiver.py", line 113, in archive_message msg.message_id_hash = self.store.add_to_list(mlist, msg) File "/home/nico/wspace/kittystore/kittystore/storm/store.py", line 92, in add_to_list list_name = unicode(mlist.fqdn_listname) AttributeError: Message instance has no attribute 'fqdn_listname'
another run give exact similar comportement.
Feel free to ask any further information
Cheers
Le 2013-12-01 11:13, Aurelien Bompard a écrit :
Oh, I forgot: I'm using patches on Mailman from all the bugs I've reported and are not yet merged:
- https://bugs.launchpad.net/mailman/+bug/1130957
- https://bugs.launchpad.net/mailman/+bug/1158721
- https://bugs.launchpad.net/mailman/+bug/1130696
- https://bugs.launchpad.net/mailman/+bug/1170347
- https://bugs.launchpad.net/mailman/+bug/1236297
All those have patches attached. There's also my work on the import21 command, but I don't think it would impact you.
This may be linked to why the archiver does not work for you. Cant you try with the paches?
Thanks. I should have thought of that earlier, sorry... :-/
Aurélien
P.S.: Hi Barry, cool to see you around :-)
2013/11/30 Barry Warsaw barry@python.org:
On Nov 30, 2013, at 08:37 PM, nicolas wrote:
I think we have to qualify a better report before sending to mailman team, i'm using r7229 of lp:mailman, and i wrote this simple mlist.py module : #-*- coding: utf-8 -*-
^^^ This isn't enough to make it work. MM3 core is written in a dialect that tries to be as close to Python 3 as possible[*]. You should put the following at the top of your module:
from __future__ import absolute_import, print_function, unicode_literals
It's possible there are some corner cases when using this as a withlist script. If so, please submit a bug report!
Cheers, -Barry
hyperkitty-devel mailing list hyperkitty-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/hyperkitty-devel
hyperkitty-devel mailing list hyperkitty-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/hyperkitty-devel
Le 2013-12-01 18:50, Aurelien Bompard a écrit :
archiver.system_archiver.archive_message(msg,mlist)There's something wrong here, the archive_message() signature is mlist ast the first argument, and msg as the second. That's what's causing the traceback.
Ok, my mistake.
Now i have something normal (except the timestamp gap where i've inserted blank line: (venv_hk)nico@vostrodeb:~/wspace/mailman$ mailman withlist -r mlist.showArchiver newsletter@lists.menestrel.local newsletter - hyperkitty <mailman.model.mailinglist.ListArchiver object at 0x22e2ad0> [12:27:31.795920] EXECUTE: 'SELECT * FROM patch WHERE version=0', () [12:27:31.806851] DONE [12:27:31.807570] EXECUTE: 'SELECT patch.version FROM patch', () [12:27:31.808319] DONE [12:27:31.811304] EXECUTE: 'SELECT patch.version FROM patch', () [12:27:31.812192] DONE
[12:30:09.609158] EXECUTE: 'SELECT list.archive_policy, list.display_name, list.name, list.subject_prefix FROM list WHERE list.name = %s', (u'newsletter@lists.menestrel.local',) [12:30:09.617901] DONE [12:30:09.619191] EXECUTE: 'INSERT INTO list (archive_policy, subject_prefix, name, display_name) VALUES (%s, %s, %s, %s) RETURNING list.name', (<ArchivePolicy.public: 2>, u'[Newsletter] ', u'newsletter@lists.menestrel.local', u'Newsletter') [12:30:09.622524] DONE [12:30:09.623575] EXECUTE: 'SELECT COUNT(*) FROM email WHERE email.list_name = %s AND email.message_id = %s', (u'newsletter@lists.menestrel.local', u'ant') [12:30:09.630188] DONE Can't get the user from Mailman: HTTP Error 404: 404 Not Found [12:30:09.662961] EXECUTE: 'INSERT INTO thread (thread_id, list_name, date_active) VALUES (%s, %s, %s) RETURNING thread.list_name, thread.thread_id', (u'MS6QLWERIJLGCRF44J7USBFDELMNT2BW', u'newsletter@lists.menestrel.local', datetime.datetime(2013, 12, 1, 18, 30, 9, 630910)) [12:30:09.667042] DONE [12:30:09.667636] EXECUTE: 'INSERT INTO email (list_name, subject, archived_date, user_id, message_id, thread_depth, content, message_id_hash, thread_id, "date", sender_email, thread_order, in_reply_to, sender_name, timezone) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING email.list_name, email.message_id', (u'newsletter@lists.menestrel.local', u'Testing the test list', datetime.datetime(2013, 12, 1, 12, 30, 9, 618457), None, u'ant', 0, u'Tests are better than no tests\nbut the water deserves to be swum.\n', u'MS6QLWERIJLGCRF44J7USBFDELMNT2BW', u'MS6QLWERIJLGCRF44J7USBFDELMNT2BW', datetime.datetime(2013, 12, 1, 18, 30, 9, 630910), u'anne@example.com', 0, None, u'anne@example.com', 0) [12:30:09.675587] DONE [12:30:09.676383] EXECUTE: 'SELECT email.archived_date, email.content, email."date", email.in_reply_to, email.list_name, email.message_id, email.message_id_hash, email.sender_email, email.sender_name, email.subject, email.thread_depth, email.thread_id, email.thread_order, email.timezone, email.user_id FROM email WHERE email.list_name = %s AND email.thread_id = %s ORDER BY email."date"', (u'newsletter@lists.menestrel.local', u'MS6QLWERIJLGCRF44J7USBFDELMNT2BW') [12:30:09.678620] DONE [12:30:09.679593] EXECUTE: 'SELECT email.archived_date, email.content, email."date", email.in_reply_to, email.list_name, email.message_id, email.message_id_hash, email.sender_email, email.sender_name, email.subject, email.thread_depth, email.thread_id, email.thread_order, email.timezone, email.user_id FROM email WHERE email.list_name = %s AND email.thread_id = %s ORDER BY email.in_reply_to IS NOT NULL, email."date" LIMIT 1', (u'newsletter@lists.menestrel.local', u'MS6QLWERIJLGCRF44J7USBFDELMNT2BW') [12:30:09.680938] DONE [12:30:09.683727] EXECUTE: 'SELECT attachment.content, attachment.content_type, attachment.counter, attachment.encoding, attachment.list_name, attachment.message_id, attachment.name, attachment."size" FROM attachment WHERE attachment.list_name = %s AND attachment.message_id = %s ORDER BY attachment.counter', (u'newsletter@lists.menestrel.local', u'ant') [12:30:09.686151] DONE [12:30:09.698702] COMMIT xid=None
which is reproduced with ease The first order (shown in log too with the migration script) will never return a record, given the table : CREATE TABLE patch ( version integer NOT NULL, CONSTRAINT patch_pkey PRIMARY KEY (version ) ) WITH ( OIDS=FALSE );
and index starting at 1.
the upgrade script always outpust : (venv_hk)nico@vostrodeb:~/wspace$ kittystore-updatedb -p hyperkitty_standalone -s settings Upgrading the database schema and populating the search index if necessary... [19:57:07.588982] EXECUTE: 'SELECT * FROM patch WHERE version=0', () [19:57:07.591484] DONE [19:57:07.592248] EXECUTE: 'SELECT patch.version FROM patch', () [19:57:07.592814] DONE [19:57:07.595901] EXECUTE: 'SELECT patch.version FROM patch', () [19:57:07.596562] DONE
[19:59:46.123198] EXECUTE: 'SELECT patch.version FROM patch ORDER BY version DESC LIMIT 1', () [19:59:46.125043] DONE Done, the current schema version is 12. [19:59:46.125824] EXECUTE: 'SELECT COUNT(*) FROM (SELECT DISTINCT email.user_id FROM email) AS "_tmp"', () [19:59:46.129038] DONE [19:59:46.129416] EXECUTE: 'SELECT DISTINCT email.user_id FROM email', () [19:59:46.130346] DONE Updating user_id fields from Mailman, this can take some time... [19:59:46.130583] EXECUTE: 'SELECT COUNT(*) FROM email', () [19:59:46.131234] DONE [19:59:46.132320] EXECUTE: 'SELECT email.archived_date, email.content, email."date", email.in_reply_to, email.list_name, email.message_id, email.message_id_hash, email.sender_email, email.sender_name, email.subject, email.thread_depth, email.thread_id, email.thread_order, email.timezone, email.user_id FROM email', () [19:59:46.133438] DONE Can't get the user from Mailman: HTTP Error 404: 404 Not Found [19:59:46.159782] COMMIT xid=None ...done!
with the same time delay
definitely something wrong with db urgrade.
Cheers
Aurélien _______________________________________________ hyperkitty-devel mailing list hyperkitty-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/hyperkitty-devel
This should now be fixed by my recent commit about the locking traceback on DB upgrade.
hyperkitty-devel@lists.fedorahosted.org
-
Aurelien Bompard -
Barry Warsaw -
nicolas＠karageuzian.com