***First Call for Papers***
*Workshop Description*
LAW-XVIII will be the 18th annual meeting endorsed by the ACL Special
Interest Group for Annotation (SIGANN). It will take place in March 2024 at
EACL in St. Julians, Malta.
Linguistic annotation of natural language corpora is the backbone of
supervised methods in both statistical and neural natural language
processing. Annotated corpora are also a major supporting source of
information for unsupervised methods, multitask learning, and evaluation of
both NLP tools and theories about language within and outside of linguistics.
The LAW-XVIII will provide a forum for presentation and discussion of
innovative research on all aspects of linguistic annotation, including
creation/evaluation of annotation schemes, methods for automatic and manual
annotation, use and evaluation of annotation software and frameworks,
representation of linguistic data and annotations, semi-supervised “human
in the loop” methods of annotation, crowd-sourcing approaches, and more.
The LAW will also provide a forum for annotation researchers to work towards
standardization, best practices, and interoperability of annotation
information and software.
*Special Theme*
The special theme of LAW-XVIII is “Annotation in the Age of Large Language
Models (LLMs).” In addition to LAW’s general topics, we specifically
invite submissions on the following topics:
– Comparison of linguistically annotated datasets vs. datasets created using
large language models. Potential topics include:
– Comparison of models that have been trained on the respective datasets
– Impact of data size of manually annotated resources already available prior
to dataset creation with LLMs
– Is synthetic dataset creation a viable option for non-standard domains,
e.g., the medical domain, where expert knowledge is required?
– Non-performance-related considerations of manual vs. synthetic dataset
creation (e.g., explainability)
– Impact and prevention of test dataset contamination in LLM training
– Usefulness of LLMs for linguistic research (in relation to annotation).
– Any other topics related to the special theme.
*Submissions*
We welcome submissions of long and short papers, posters, and demonstrations
relating to the special theme or any aspect of linguistic annotation,
including:
– Annotation procedures
– Innovative automated and manual strategies for annotation
– Machine learning and knowledge-based methods for automation of corpus
annotation
– Creation, maintenance, and interactive exploration of annotation structures
and annotated data
– Annotation evaluation
– Inter-annotator agreement and other evaluation metrics and strategies
– Qualitative evaluation of linguistic representations
– Innovative means to evaluate annotation quality
– Annotation access and use
– Representation formats/structures for annotations of different phenomena,
especially annotations at multiple levels, and means to explore/manipulate
them
– Linguistic considerations for merging annotations of distinct phenomena
– Annotation schemes, guidelines and standards
– New and innovative annotation schemes, comparison of annotation schemes
– Methodologies and resources for annotation scheme development
– Best practices for annotation procedures and/or development and
documentation of annotation schemes
– Interoperability of annotation formats and/or frameworks among different
systems as well as different tasks, frameworks, modalities, and languages
– Results from the application and evaluation of standards for linguistic
annotation
– Annotation software and frameworks
– Development, evaluation and/or innovative use of annotation software
frameworks
Submissions should report original and unpublished research on topics of
interest to the workshop. We also invite substantiated position papers, in
particular with regard to our special theme. Accepted papers are expected to
be presented at the workshop and will be published in the workshop
proceedings. They should emphasize obtained results rather than intended
work, and should indicate clearly the state of completion of the reported
results.
A paper accepted for presentation at the workshop must not be or have been
presented at any other meeting with publicly available proceedings.
Long/short paper submissions must use the official ACL style templates. Long
papers must not exceed eight (8) pages of content. Short papers and
demonstration papers must not exceed four (4) pages of content. References do
not count against these limits.
Note: The supplementary material does not count towards page limit and should
not be included in the paper, but should be submitted separately using the
appropriate field on the submission website. All submissions must be in PDF
format.
Reviewing of papers will be double-blind. Therefore, the paper must not
include the authors’ names and affiliations or self-references that reveal
the authors’ identity–e.g., “We previously showed (Smith, 1991) …”
should be replaced with citations such as “Smith (1991) previously showed
…”. Papers that do not conform to these requirements will be rejected
without review.
Authors of papers that have been or will be submitted to other meetings or
publications must provide this information to the workshop co-chairs
(law-xviii-2024@googlegroups.com). Authors of accepted papers must notify
the program chairs within 10 days of acceptance if the paper is withdrawn for
any reason.
We follow previous and current ACL policy to establish an anonymity period
(from submission to author notification) during which non-anonymous posting
of preprints is not allowed. Also included in that policy are instructions to
reviewers to not rate papers down for not citing recent preprints. Authors
are asked to cite published versions of papers instead of preprint versions
when possible.
Papers can be submitted at https://softconf.com/eacl2024/LAW-XVIII/.
If you have any questions, please feel free to contact the program co-chairs
via e-mail or check the workshop website
(https://sigann.github.io/LAW-XVIII-2024/) for updates.
*Dates*
(All submission deadlines are 11:59 p.m. UTC-12:00 “anywhere on Earth”)
Anonymity period starts: November 18, 2023
Submission of long and short papers: December 18, 2023
ARR Commitment deadline: January 17, 2024
Notification of acceptance: January 20, 2024
Camera-ready papers due: January 30, 2024
Workshop: March 21 or 22, 2024
*Workshop Organizers*
Manfred Stede (Program Co-Chair)
Sophie Henning (Program Co-Chair)
Amir Zeldes (ACL SIGANN President)
Ines Rehbein (ACL SIGANN Secretary)
*Contact*
Website: https://sigann.github.io/LAW-XVIII-2024/
Submission: https://softconf.com/eacl2024/LAW-XVIII/
E-mail: law-xviii-2024@googlegroups.com