Over the past several months, I’ve been reflecting on how I’m living my life, both as as librarian and as a person trying to live a more complete, fulfilling, meaningful, and positive life. The emphasis on the positive has been both intentional and fairly exhausting as I create new habits to seek information about the world that goes beyond sensational headlines. It’s utterly depressing how much of the daily news and information we encounter is negative. I’m choosing here to share back the positive I encounter, particularly the great good that people do and create in the face of horrible things.
Part of what I’ll be sharing comes from what I’ve read and listened to each week, along with my excerpts and insights. My motivation is partly selfish, in that I want to be more mindful and reflective about the information and stories I consume. But I also want to share the wonderful voices a perspectives that have enriched my life. Here we go!
On Being (one of my go-to weekly resources for inspiration and positivity) had a fantastic episode this week – How trauma lodges in the body with Bessel van der Kolk. The notion that trauma (and chronic stress, I think) are experienced in the body as well as the mind opens up avenues of treatment for many of the physical manifestations of our lifestyles. As I think about the millions of refugees, black Americans, Muslim Americans, poor women and other folks experiencing a daily onslaught of stress and threat to their very existence I wonder how we can help them to make sense of these experiences, even as they continue to experience stress and trauma. Is it even possible to begin processing and dealing with trauma that is ongoing? Yoga and rolfing and EMDR sound like promising alternatives to medication, particularly medication that doesn’t work or causes side effects to be treated with other medications.
Brain Pickings is another excellent blog that exposes me to writers and ideas I wouldn’t typically encounter. Personally, I tend to obsess about time in unhealthy ways, so the recent post about Herman Hesse resonated. A colleague recently shared this interview with the authors of The Slow Professor: Challenging the Culture of Speed in the Academy. As someone who is very goal-driven and interested in practical, applied research, I’m unclear how this applies to my own work. But I think higher ed and industry need to get better at recognizing the value in diverse models for creating knowledge and exploring the world. Basic science gave us penicillin, electricity, and graphene, to name just a few. Applied research that examines our systems of government, education, healthcare, transportation, among others is also crucial, for different reasons.
“The high value put upon every minute of time, the idea of hurry-hurry as the most important objective of living, is unquestionably the most dangerous enemy of joy.”
Yesterday, I finished reading Ta-nehisi Coates’s Between the World and Me. I don’t have anything to say about this that hasn’t already been said more eloquently by others. It was eye-opening and visceral, more even than I anticipated. I simply don’t know a way forward – how we can acknowledge our past wrongs as a country while talking about how to build a better future that includes everyone in this country. As I try to fill the gaps in my formal education, particularly about race and class in America, I’m reading to help me understand the history of the country we really live in. Next up is Waking Up White by Debby Irving. Thanks to Twitter, I’ve discovered voices unlike any I heard growing up, even in a relatively diverse family. Writers like Ijeoma Oluo, who writes at The Establishment and The Guardian, open my eyes to injustices that exist in my country, in my own community.
Professionally, I tend to read several books at once, annotating and using them to answer specific questions or needs relevant to my work. Working on a lit review of program evaluation and service assessment in libraries, I came across a fascinating book that I’m slowly absorbing – Researching your own practice: The principles of noticing. It’s written for teachers, but so much of it has been applicable to librarianship. The professional practice of noticing is rooted in Eastern practices like Zen mindfulness or presence, which is a practice I’ve been cultivating. It was surprising to think about practicing mindfulness in the classroom, but I’m going to try it out the next time I teach a workshop. I can see how the practice can also be useful for reference and consultation interviews and meetings.
Onto parenting! One of my favorite and creatively inspiring blogs is Posie Gets Cozy. She is far more creative than I could ever be, plus I’m not a sewer, so this blog doesn’t feed my tendency to do all the things! . I can just appreciate the pictures and stories. She does have a crochet pattern called Honey Bunny that is adorable. My family photos are nowhere near as good, but hers inspire me to see the beauty in our everyday lives – in waking with up with E in the middle of the night, in making dinner at the end of a long day, in trying to work as I do 5 loads of laundry.
Something I’ve been pondering over the last few days. How do we invite people to be their whole selves at work and school, instead of asking them to leave pieces of them behind?
There were so many thoughts and things I wanted to say during my talk on the OpenCon Research Evaluation panel but I just didn’t have the time to include them all. Here I’m filling in some of the gaps of the talk I actually gave a bit.
My thanks to Nick for inviting me to be here today. I’m so honored to speak to you all and to be a part of this panel. I have gained so much from the OpenCon community that I’m thrilled to be able to give back.
Let’s end on a note of hope. We talk a lot about what is wrong with the scholarly communication system, but I’d like to take a few moments to envision a better way.
First, I have to acknowledge the privilege of my position. I work in the US, in a high research activity university, on a unique campus that supports innovation, in a well resourced library, with administrative support for openness.
I was invited to speak to you today because I work at a rather unique place. Civic engagement has been a part of our campus mission since it’s inception. This means that we don’t function in isolation – we are a part of the community. We need to work with the community, along side the community members. This emphasis creates an atmosphere that is perhaps more willing to embrace open.
While it is really important to have champions, but those values need to be encoded in policies and institutional practices in order to enable sustainable change.
These are some of the ways that we have tried to encode openness in our campus policies and practices.
Engagement in faculty governance has led to addition of OA as a campus value and we are collaborating with specific schools and departments to revise guidelines at those levels. Our education efforts include workshops & consultations to help faculty make a stronger case for promotion and tenure. We have been able to change practice by modelling the practices we advocate for, openly discussing how those practices are affecting our own professional advancement, and by establishing an Open Access Fund.
Our current practices emphasize the value of research product, rather than valuing all of the work that we do as scholar – including teaching and service.
To successfully advocate for the values we want to embed in scholarly dissemination, the dossier should function as a communication tool – the story you tell should embody the values you hold.
The choices that we make about how we communicate and disseminate our work should reflect our values. We can be more strategic about planning for dissemination and impact. Think more broadly – who are you trying to impact? who are you trying to communicate with? A clearer understanding of these goals makes us better communicators.
As I help faculty develop their dossiers, I tried to break it down into simple categories. Good scientific communication principles emphasize the first three, but I think we also need to keep in mind the type of impact we are trying to achieve. The next time you think about your work, think about how to create or contribute to the dialog.
When you think about the types of impact your work can have, there are a couple of helpful resources. The first is ImpactStory’s “flavors of impact” concept. The second is the Becker Model for Assessment of Research Impact.
While this came out of a medical research environment, it can be translated to non-medical research. For example, at IUPUI I adapt the clinical implementation category to a campus value and initiative called TRIP – translating research into practice.
Right now, promotion and tenure is largely based on articles and citations.
Until a few years ago perhaps, we were dependent on the linkages between these very static and narrow products (articles, books, presentations).
In this model, narrative statements focus on quality, reputation, prestige. This way of valuing scholarship is very self-referential and closed.
We are in a transition period. The types of scholarly products that “count” is expanding.
As are the types of metrics available, but we are still learning what these metrics data mean. Too often, their use is driven by sheer availability, rather than the strategic and thoughtful dissemination.
We have lots of data, but data and metrics are not the solution. This is a cultural problem. Metrics can be a great way to start the conversation.
I believe a better way for scholars to demonstrate the impact of their scholarship is by taking a program evaluation approach. This means identifying the products created, the stakeholders or audience for those products, and the desired short-term, intermediate, and maybe even long-term outcomes. When a strategic plan is created, it enables a scholar to gather, present, and understand how specific evidence (including metrics) can be used to demonstrate the impact of their work.
At my own institution, a great example of community engaged research with direct community benefit comes from our school of public health
Silvia Bigatti: Your Life. Your Story Latino Youth Summit
This is a powerful story that demonstrates the impact of her research, teaching, and professional practice, but our current system does not really provide guidance in how to document this type of impact, particularly the face-to-face, personal contact.
Her work is beginning to get local and statewide news coverage. How can this faculty member tell the story of this project? How can we help her demonstrate impact that isn’t captured by metrics?
First and foremost, the dossier is the story of a scholar. Metrics and evidence are great, but the story needs to come first. As we support faculty in developing dossiers that tell their story and document the impact that their work has had on the community. These are some of the ways I think that we can change how dossiers can communicate diverse impact and advocate for greater openness in research.
Open is a means to an end, not the ultimate goal. Open is a set of practices that promote the values we want our scholarly ecosystem to embody.
TAKE ACTION: Like Ahmed said, change is hard and it may take time to for your impact to be felt or seen.
I hope these questions can stimulate our discussion about how to improve research evaluation.
One of my biggest professional challenges is estimating how much time it will take to complete tasks. This challenge is hugely amplified when it comes to projects that take more than a month or involve other people. Surprisingly, I never learned this project management skill as a lab/research coordinator. I think it’s primarily because the investigators I worked for didn’t really use project management approaches and the recognition that we as a team were over committed would not have changed anything. Typically, we all just worked longer hours to get the work done, or found a way to hire another RA. This no longer works for me, for a lot of reasons.
It’s time to quantify how much time is actually available to do deep work each week and semester. I hope that showing my math will make this process more real and perhaps help me stick to a realistic work schedule. Since I’ll be teaching an informatics course on overload in the spring, I’ll be devoting one evening and a few hours on weekends to that. I really don’t want to work at home more than that.
Weekly Time Budget
40 hours (An optimistic goal?)
– 8 hours for administrative meetings, tasks, email, & scheduling (sad, but true)
– 2 hours for planning, monitoring productivity, & reporting (both for Library Admin & myself)
– 16 hours for Research Data Services (workshops, consultations, outreach, events, content development, reading, projects, etc.)
– 8 hours for Research Metrics Services (workshops, consultations, outreach, events, content development, reading, projects, etc.)
– 2 hours for collection development
– 4 hours for service activities
So, out of a 40-hour work week, it looks like I should budget 24 hours to get actual work done – you know, thinking, creating, developing, teaching, coordinating, etc. That’s kind of depressing, but feels fairly realistic as I glance back over my calendar for 2016. Since I’m actually trying to avoid the workload I’ve had this year, I cut back service to about half.
*This adds up to about 256 hours per semester for data services and 128 hours per semester for metrics services. Those numbers sound incredibly low, which means I’ve been inflating how much time I estimate is available to develop new content and events. No wonder I’m exhausted after a semester of 7 workshops, most of which were new or significantly revised, plus two conference presentations, putting the final touches on two publications, in addition to the unplanned stuff like consults, meeting with new librarians, etc. I think these might be the more salient numbers when it comes to developing my semester plan.
Getting things done is also about maintaining focus, which boils down to managing my energy. I tried to factor that into the baseline hours available for deep work, but will need to see what looks like from day-to-day. Only time will tell how accurate that is over the spring semester.
*Added later on 11/27/16
It has been a wonderful Thanksgiving and I am grateful to have some down time to reflect and rest.
It’s that time of the year when we begin discussing the annual review and I begin to reflect back on how things have gone this far. I still have a series of Deadlines and workshops before the semester winds down, so I’m taking this weekend to learn from my mistakes in 2016. It seems I made many of the same old mistakes, along with a few new ones.
Here’s what I’ll be considering as I outline my annual review and develop my plan for 2017.
- What goals have and have not been accomplished? How do these align with my priorities for the year?
- Am I managing my time and energy realistically?
- Is my current task management system working? Is it preventing me from over committing? Does it help me estimate time to completion? Does it facilitate setting weekly and daily priorities and task lists?
- What internal and external challenges do I face in balancing work/home/personal needs?
- What experiences from 2016 do I value most? What situations from 2016 would I like to prevent in 2017?
Last month I gave a presentation to an intro to research ethics course. It’s one of two offered by the IU School of Medicine to fulfill training requirements of the NSF and NIH for residents, postdocs, fellows, etc. The presentation differed a bit from my usual RDM talk in that I was able to highlight the ethical responsibilities rather than emphasizing just the personal benefits of good data practices.
Not surprisingly, when we started to discuss openness, the problem of scooping came up. Two of the three students who mentioned it were bench scientists, while the third was a surgical resident. I asked if they actually knew someone who had been scooped. One of the bench scientists said yes, while the surgical resident described a story about a surgical technique that was invented by one person while another got the credit. As I’ve talked to my library colleagues, I began to realize that many librarians may not be comfortable challenging these assumptions or offering counter arguments. I’d like to share my responses to common fears or excuses for adhering to the status-quo in research, hoping that others will add their own arguments. These are fairly rough and I’d like to strengthen and focus them, so please share any (constructive) suggestions!
- What does being scooped even mean?
If another lab beats you to publication, your results are not suddenly invalid. Replication and reproduction are crucial to building a strong base of scientific evidence. Yes, some publishers and researchers do not hold these values, but those attitudes can (and should) change. Frankly, if another investigator has the same idea and does it better than you, that is a net win for science. Consider how you can learn from them, collaborate, or contribute in a different area. No one wins all the time. Do it better next time.
- How often does scooping really happen?
Not very. Sadly, this is based on “anecdata” for now since answering this question is really, really hard.
- How does being “scooped” hurt you?
Most often, this boils down to people being worried about not getting credit for their work. If being first in your field matters most, then controlled and early sharing of your results, rather than waiting until you have enough evidence to get into Nature/Science/NEJM, can actually help your career. Yes, how you do that controlled sharing varies by discipline and community of practice, so there is not one right answer. Some easy things to do are put your data/code/procedure somewhere really durable (like an institutional and/or subject repository) and get a DOI (digital object identifier) assigned so it can be easily cited. The patent process is a great example of the controlled release of information to protect IP rights. In exchange for exclusive rights to produce a product, you have to release information about how it is made.
- An aside: Valuing being first over being accurate is seriously messed up. Science is about the accumulation of evidence to support a theory that explains how/why things happen.
- Research is about the process, not the end result.
The point of research, particularly scientific research, is that is allows us to correct our mistakes over time, as the body of evidence grows and our methods/technology improve. Each of us conducting research today is building on the work of millions who came before us. You are part of a conversation and a community. Advancing knowledge is not about you and your career.
- Incentives currently reward the results of research, rather than a sound process for getting those results.
We need to fix this. See the myriad of stories about retractions, data fabrication, outright fraud, and hidden clinical trials with negative or null results. This is how the Vioxx mess happened.
This interview with Elizabeth Pisani about her recent BMJ article titled “Beyond open data: Realising the health benefits of sharing data” is a fantastic and honest conversation about the ambivalence many researchers feel about sharing their data.
As a self-proclaimed advocate for open research, I decided to apply that ethos to promotion and tenure. As I started preparing my dossier in earnest last fall, I began to understand why so many faculty get overwhelmed and confused when they make decisions about where to publish, which journals to review for, and how to talk about their work. Despite excellent institutional programming and support, faculty often receive conflicting and vague advice. Combine this with the lack of transparency about how to actually demonstrate impact and it’s no wonder faculty are hesitant to make publishing and dissemination choices that challenge the perceived status quo. Librarians on the tenure-track suffer from this too. I decided I could help in a small way by openly sharing the strategy, tools, and examples from my own dossier. In the end, I redacted a few things from my appendices that relate to other faculty grant proposals. Otherwise, it’s all out in the open. I also developed some tools to help me manage the process, which I’ll share in a later post. Putting together a dossier requires some serious project management strategery!
Dossier files in Figshare (link corrected) & IUPUI ScholarWorks (pending)
There is a lot of implicit institutional context embedded in the structure and content of dossiers, so there are a few things to keep in mind should you decide to take a deep dive into these files.
- The structure and content are set forth in the IUPUI Promotion & Tenure Guidelines, while the standards by which I will be evaluated are in the IUPUI Librarian Standards for Performance, Professional Development, and Service. These are not posted publicly, but I am requesting permission to share them here.
- I also relied on my colleagues who were generous enough to share their documents with me. Many of them are not comfortable sharing their files publicly, but our Office of Academic Affairs does maintain a variety of very useful sample dossiers on their website.
The criteria I have to demonstrate that I have satisfied for promotion and tenure are:
- Evaluations cover the areas of performance, professional development, and service.
- For tenure, performance must be excellent, and professional development and service must be satisfactory.
- For promotion from assistant to associate librarian, performance must be excellent, and the candidate must demonstrate a level of achievement beyond satisfactory in one of the other two areas. The third area must be satisfactory.
As I prepared my dossier and reflected on the process for the FORCE16 presentation, I realized a couple of things that no one had discussed with me in all the conversations and workshops I’ve been involved in.
- The candidate’s sections are just a part of the dossier. For me, sections 6-11.
- 6: Candidate Statement
- 7: Performance (primary criterion)
- 8: Professional Development (secondary criterion)
- 9: Service
- 10: Curriculum Vitae
- 11: Appendices
- Candidates are being encouraged to make judgments of impact about their own work when that is truly the role of the reviewers, particularly external peers and Library colleagues.
In terms of the story I wanted to tell about myself, discussions with colleagues outside the Center were tremendously helpful. Figuring out how to introduce my work at a high level in the candidate statement, then expounding upon specific projects within the three sections was more difficult than I expected. The outlines I developed in the fall ended up changing drastically; once again, I learned that writing early and often is a better strategy than over planning.
- Candidate statement
- Performance – I ended up describing what I do as research support. This includes
- Research Data Services program
- helping faculty meet funder & publisher data requirements
- data management curriculum & training
- enabling data preservation, sharing, & reuse
- developing a research data policy for IU
- research metrics services
- consultations supporting faculty in making informed decisions about publishing, dissemination, and reviewing
- liaisonship to the School of Public Health
- Research Data Services program
- Professional Development
- evidence-based practice
- research data services in academic libraries
- practical application of research metrics
- content analysis of successful dossiers
- Campus (IUPUI) & University (IU)
If you are primarily interested in how I presented metrics to demonstrate impact, check out section 8 (Professional Development) which has pages of summary tables and graphs.